Skip to main content

Code versus Configuration


At Ethos we are building a distributed mortgage origination system and in mortgage there is a lot of
different user types with processes that vary depending on geography.  One of our ongoing discussions is about how much of the logic resides in code vs. being in a workflow system or configuration.  After researching this topic for a bit, I have arrived at a conclusion that the logic should live outside of code very infrequently, which might come as a surprise to a lot of enterprise software engineers.

Costs of configuration files and workflow engines

First thing that I assume is true is that having any logic outside of the code has costs associated with it.  Debugging highly configurable system involves not only getting the appropriate branch from source control, you also need to make sure that the right configuration values or the database.  In most cases this is harder for programmers to deal with.  In many FinTech companies where the production data is not made readily accessible to developers this provides additional barrier to solving customer issues.

Writing unit tests also becomes harder, to spin up a unit tests the data living outside the code needs to be populated.  With an elaborate configuration you also need to test the different permutations that the data can take on, which can very extensive.

IT Administration

There are systems that need to support a variety of different workflows depending on the organization or even a particular transaction.  A good example of such a system is a CRM.  CRM by itself is generally a customer database with a nice interface, but sales and support patterns vary greatly across organizations.  The type of data that is being tracked is also very different among the different customers.  A big audience in a system like that is an IT administrator.  CRM, ERP, LMS and other categories assume that upon purchasing a “platform” you will have someone go in and configure it to fit a particular business need.

Systems that have this type of flexibility requirements cannot have all the logical forks in code, there is a need to create an interface for administrators to configure the system.  If you have this flexibility and a nice, well documented user interface to work with it then the system gains flexibility necessary to support a variety of different scenarios.

The key in these scenarios is to know the true cost of providing every configuration switch or workflow support: documentation, debugging, export/import, permissions and versioning of such a configuration are probably going to be needed.  A lot of people come short of this, they create a configuration file or a workflow database table, but no way to work with it or support it.  As a result, you pay all the costs but don’t get the benefits: the only people who can configure such a system are the developers who wrote the code that processes the configuration.

SaaS vs On-Prem

Some systems have a requirement to be on premise.  When an upgrade to the logic requires someone to take the system down, upgrade the logic and possibly migrate the data, it might make sense to take some of the logic out of the code into a configuration file.  Managing different versions of the solution that might have different business processes could be very hard, you might need to provide the configuration, import/export, admin interfaces and debugging to such a system anyway.

Systems that are deployed and managed by the vendor “in the cloud” can get away without providing an administrative interface.  The engineers can update the code when the business requirements change, given that it is not scenario specific.  There is also no need to support multiple versions of the code when you deploy it to your own infrastructure.

How frequently does the workflow change?

One of the key data points on configuration vs code is the frequency when the logic changes.  Some logic just doesn’t change that frequently.  Here is an example: password reset and new account setup are scenarios that are well understood and rarely change.  The content and design of the e-mails might be different and might also be stylized based on what tenant is being used.  This is a case where most of the engineers split the configuration vs code in the right place: code for handling the reset/new account logic and configuration for the e-mail template.  The idea is that the password reset logic is not changing that frequently, but the e-mail template might.

What technology is used?

Last piece of the decision is the technology that is being used.  Systems that are based on interpret languages often ship as source code and there is no reason to create a separate set of files to configure it.  A famous example of such a system in settings.py for Django.  The variables for settings can be right inside the Python code file without a need to create a config file.  Settings file can be extended to take environment variables for some settings that are expected to change such as debug level, but not for everything.

Interestingly enough some of the compiled languages like C# have moved to a deployment model where you do not need to compile them, you can just issue ‘dotnet run’ and the program is going to compile and run without having an explicit compilation step.

If you are using a technology stack that doesn’t require you to compile into binaries (or if that happens behind the scenes), it might be just as easy to keep the configuration values, and settings in a code file in a known location.

Summary

Given the tax of configuration it’s very important to ensure that you weigh pros and cons of supporting logic external to code in your system.  I’ve seen people too many times err on the side of providing an extensible system and ending up in a system that was slow to develop, impossible to debug and hard to manage.  The configuration points provided were never used by IT Administrators because the administrative interfaces were poorly documented and the engineers constantly had to get into customer data to make sense out of what the system was using.

Configuration and workflow are powerful tools, but just like any powerful tool it’s easy to shoot yourself in the foot.

Some references
https://blog.urth.org/2011/01/06/config-versus-code/
https://medium.com/@nimesh.mittal/when-to-use-workflow-engine-9a0c20442978 

Post a Comment

Popular posts from this blog

SDET / QA Engineer Interview Checklist

After interviewing and hiring hundreds of engineers over the past 12+  years I have come up with a few checklists.  I wanted to share one of those with you so you could conduct comprehensive interviews of QA Engineers for your team.

I use this checklist when I review incoming resumes and during the interview.  It keeps me from missing areas that ensure a good team and technology fit.  I hope you make good use of them.  If you think there are good questions or topics that I have missed - get in touch with me!


SDE/T or QA Engineer interview checklist from Mike Borozdin
If you like this checklist you might want to check out these posts:
Emotional Intelligence in Software Teams  and Good-bye manual tester, hello crowdsourcing!

Pull Requests and Code Reviews

Software development involves a great deal of collaboration.  One of the most basic blocks of collaboration on a software development team is a code review.  There have been many different ways of doing code reviews over time, some of this has been dictated by the tools available.  Git and online source collaboration tools created a set of best practices that are worthwhile of adopting on any team.

About a month ago I have looked at various articles about how to best create a Pull Request (PR) and do a code review and the attached presentation is the result of this research.  The presentation can help you guide your team and develop a set of collaboration practices that works for your particular situation.

It’s good to start out with why to seek a code review.  Having clarity about your intentions helps you guide the person helping you with code reviews and also to manage your expectations about you can get out of the code review.  The reasons for seeking a code review are generally …

Why you should take the software job in San Francisco (or not).

Silicon Valley is an iconic place for technology.  Many people say this is the place for the “best and the brightest.”. Apple, Google, Facebook, Salesforce, Twitter and other top companies draw a lot of talent form all over the world and the largest chunk of VC capital goes to companies in the Bay Area, so it seems like moving here is a no brainer!

The real situation is actually not that simple, I believe there are three scenarios where it makes sense, but in many cases living in the Bay yields disappointing results.  The cost of living, housing situation, homeless catastrophe make places like San Francisco a lot less appealing to a lot of people.  So in what situations does it make sense to move to SF?

Startup founder raising millions There are many places to be a startup founder, but if you are looking to raise capital the largest pool of VC money is in the Bay Area.  There is an established network, events and conferences which give founders an opportunity to pitch more people th…