Tech Talk: "How to build and maintain 1.000 Apps" mit Michael Schöndorfer von hello again

Tech Talk: "How to build and maintain 1.000 Apps" mit Michael Schöndorfer von hello again

Hello, my name is Michael Schöndorfer, I’m the CTO at hello again. Today I want to share with you how we are trying to achieve our next goal of building and maintaining 1.000 apps, where we are pretty much halfway through – afterwards of course going for the next milestone!

First I’d like to start with explaining you a little bit of what we do at hello again. In general we want to make our clients or customers more valuable. To give you an idea, this is usually resulting in customer loyalty solution – most likely an app where you can have a bonus system and collect points, just to give you an idea of what we provide. What differentiates us from a lot of our competitors is that every client gets their own customized solution. Instead of having one big hello again app, where all the data is shared with everybody, we really want to build communities and keep the data in our client’s hand.

Who are our customers? To give you some examples, we have a lot of clients in retail stores, also online-shops. We also provide solutions for the service industry – for example, hairdressers is a very good example – but also economic regions like cities. The size of our clients varies quite a lot, from having a few thousand users in their community to a few million users.

To give you an idea of our featureset of what we provide: our apps usually have a user management, you can have different loyalty mechanics to collect points – including gamification. For example, for purchases you get a certain number of points. On the right side you see one of our client’s apps – Müller, a drugstore chain. Here you can collect Müller Blüten and redeem them for rewards when purchasing again. We are trying to incorporate user feedback – we do that by providing polls, you can get user ratings, you have different forms and also potentially a chat feature. What adds to the complexity is, we provide integration to third party services – for example booking tools for hairdressers – and we also provide sharing features with services like facebook or WhatsApp. We also introduced payments now. And one of our core features is messaging – to send push notifications or emails also via automated actions. For example at your birthday you get a certain number of points and a message via push-notification. Also another big topic for us is analytics and reporting.

What we take care of in the whole process of creating a customer loyalty solutions is, we develop the mobile- and/or web-app, we also do the testing – which adds a lot to our complexity – and we also do the App Store release management, which has to be done individually for each app. We also provide a CRM, where our clients can manage their users. This also includes – as I said before – marketing automation and statistics. Here you can see a screenshot of our dashboard, which gives you a rough overview of the current statistics. We also not only do server development, we take care of the infrastructure and do the maintenance.

All of this results in certain challenges, compared to providing a different architecture – for example if you’d only have one app, or we would have a smaller set of clients with a bigger number of users. What we have to do is update a lot of individual apps! That means submitting those apps to the app stores and other challenges that come with it, which I will go into detail later. You have to manage the complexity overall, with a big featureset, with different clients, different branches and different sizes. It’s easy to introduce regressions for other apps, if you work on a certain feature, which has to be tested all the time. And it is sometimes difficult to optimize for all different usecases – for example you develop a feature for a client that has only 1.000 users, you still have to optimize that, in case a client with 2 million users uses the same featureset.

To give you an idea how we built our whole architecture: in the core is our hello again server, it runs multiple instances of our back end, but they are all multi tenant capable – that means we can scale quite easily, if we need more resources. For example, we have a cyber monday or black shopping week, where really a lot of resources are required, we can easily scale with the Google Cloud services automatically. We have to integrate into different communication gateways, like email, push, others are sometimes client-specific services. We provide a dashboard – as I mentioned before. And then we of course have the client-facing front end parts where we have different mobile apps, that run for each user. We also provide a web app solution, that is basically a trimmed-down version of the app.

So, how do we achieve this goal of providing so many different solutions, tailored to each and every client, but still being able to do this without 10.000 developers? First of all, our server back end is multi tenant capable. That means we can run the same instances multiple times and we don’t have to do it individually for each client. That also means we do not provide on-premise hosting and therefore we have to say „no“ quite often if certain potential clients request it. All of this makes releasing a lot easier, and a lot harder. It’s easier because we only have to update one server. It’s harder, because you automatically update the server for all apps and you cannot test each and every app combination every day when we do a release. That means we need a lot of automated tests. We can also not have any downtime, because updates for one client’s usecases should not affect a different app. Also, one big part of our core-competency is the modular app system, which I’ll explain later. We also decided to develop with React Native, so we don’t have to develop both versions of an app. And it helps us also to have a very JavaScript-heavy codebase and our developers can hop quite easily between the components. We also did a lot of automation, especially with continuous delivery, but also with setting up certain processes, when creating the apps and so on. Another big part here is our app builder, which helps us to create our solutions without having to code a single line in most usecases and it’s non-developer friendly, where our project-management team can handle most of the tasks. We also use CodePush to do JavaScript only updates, therefore we can bypass the App Store. And we try to provide easy integrations for other developers, so in case very client-specific requirements are needed, we can outsource this to other developers or to the client themselves, for example via WebHooks.

Here is a diagram on how we built our modularized app framework. On the basis we have our app framework that is used for each and every app. On top of that we have certain extensions: one extension that is used every time is the authentication extension, most of the time we also use the loyalty extension. Then we have certain featuresets that can be used depending on the client – for example a chat feature – as well as very client-specific extensions that should not interfere with any other code. On top of that, we have certain configurations. So an authentication extension can look very different for one app than for the other app. For example, which login providers you use – email and password, or a phone number.

Internally, in the app we use Redux for communication between extensions that forces us to have very good internal APIs. To give you an idea what an extension is: an extension can provide different screens, certain Redux actions, it also can bundle certain native and React Native dependencies and it can listen to certain events to do specific customer or generic actions.

That also means for example, if we need the facebook SDK to do the facebook login – in case a client doesn’t need that feature – we can just extract it into its own extension and it doesn’t bundle in the whole app.

When building the app, we then fetch the list of extensions that are necessary, all the configurations and only build what is necessary and therefore we can get rid of all the client-specific code in the common extensions.

This then heavily requires certain configuration items, we have different levels – for example, we have the configuration on the build time, where all the navigation structure is already baked into the app, also the theming, or things like a startup screen. Then we have certain runtime configurations – on the server that could mean how many points do I get for registration, how many check-ins do I need for a certain challenge to succeed. On the app this could mean on the tutorial slides, what text do I have there, how many tutorial slides do I have, or nothing at all. We could introduce a maintenance mode or for the signup process, which fields do I even provide. These are all things we can then update without having to release a new version of the app during runtime.

Then there’s also obviously certain dynamic content like rewards, news articles, events – that is fetched depending on the client.

Our apps are also fully translatable, that helps us individualize them as well for the different clients.

What I’d like to share with you are some lessons we’ve learned from this approach. One – maybe obvious – might be that automation can be really hard. A lot harder than we originally anticipated for certain things, but in an environment like this or in our context it usually pays off. For updating apps, there are certain pitfalls that we didn’t anticipate – for example, every few months you have to accept the license agreements for Apple. This requires two-factor authentication and we would have to call the client every time. So we built our own system with physical phones that get the two-factor authentication codes and automatically do the manual labour tasks to a certain extent. For testing apps, we found out that the firebase app distribution is a really useful tool to let our clients test the apps beforehand. And in case the clients can’t even do that because they don’t have a phone – or only have an iPhone and not an Android phone – we built an app simulator into our dashboard where they can directly in the dashboard test the newest version of their app. Something we are really investing heavily is an app automation framework that works for us, where we can test all the different variation of our apps automatically. And – something that might be also very obvious, but payed off quite a lot for us – we have high code coverage for automated tests, so that we don’t have to do manual tests for each and every case when we’re releasing a new server version. We also do regular external penetration tests – that is something that is also crucial, as we have so many different usecases and variations and you cannot think of every scenario on your own. So, external personell can really dig deep and look into what are some backdoors and help you strengthen your system. And, you’re really dependent on third-party services. If Google says a certain SDK version is required now, you have to update each and every app in the next two weeks for example – where our architecture helps quite a lot, but that also means we have to update every app.

Some of the downsides of our approach: it can be very difficult sometimes to change even small things and therefore it forces us to say „no“ sometimes. The complexity increases exponentially and you always have to think of which usecase you’re affecting with this line of code, which might be not the case if you have implemented an individual app. As I said, we need to update every app individually and there’s still certain setup tasks required.

Last but not least, I want to explain to you the advantages of this approach: we don’t need any native developer, all our front end code is JavaScript only – so all the all the JavaScript developers out there, we’re looking for new developers! There’s no need to run several instances of the server, we can update them at once and also share resources a lot better. And our apps are still optimized to a certain extent, code-wise and size-wise, because we only need certain extension for each and every app. Every bugfix and improvement we implement in one app benefits all other apps as well. And it forces you to make clear internal APIs, which help to create a better architecture overall and helps you on the long run with maintenance.

Lastly, I wanna say thank you for this opportunity! We are really, really looking for people that help us support this architecture I explained. And help us reach our goal of 1.000 – or soon more – apps.

Particularly we are looking for teamleads, for server development, for front end development and in general for back end developers with Python – front end developers should be JavaScript aware, React Native and Vue.js.

Thank you very much!



Erfahre mehr zum DevTeam von hello again GmbH