Terence Bennett - February 13, 2016

Imagine that you are creating a Windows desktop application. You design the APIs that you need, and send the specification off to Microsoft. They build the interfaces, and deliver them months later with Windows Update. Now you can finish your application, but if any of the APIs aren’t just right, then you have to ask for changes. Imagine how much time the design process and correspondence with Microsoft would take. Imagine how bloated and insecure the Windows operating system would become with all of the custom APIs requested by different developers.

Incredibly, this is exactly how client and server teams build REST APIs today. And the results are similar: they build new REST APIs for each new project. The back and forth interface negotiation takes forever. Server-side complexity increases until the system grinds to a halt. By the way, REST APIs for mobile applications are externally facing, so each one of them is a potential new attack vector. This expensive and time-consuming effort results in an overly complex backend infrastructure that is not reliable, portable, scalable, or secure.

In this post, we’ll cover:

Of course, the imaginary scenario described above doesn’t happen in the real world, because Microsoft has created a PLATFORM for application developers. A developer can build almost anything they want without contacting Microsoft at all. So what would happen if someone created a platform for REST APIs? It’s not easy, I’ve been working on that problem for 10 years, but it’s not as hard as creating the Windows operating system.

Do you know what ODBC and JDBC are? They are drivers that middleware applications use to talk to a database. Do you know how often these pieces of software are modified for a particular application? Almost never. If you tell your head of engineering that you need to rewrite the ODBC or JDBC driver he or she will call you into their office for a full explanation. This is because these drivers are robust enough to support an absolutely gigantic number of long-tail application designs. Why can’t a REST API Platform work the same way?

Right about now, the professional software developers reading this might be angry with me. I’m sorry, I know that building REST APIs was fun. Building entire web sites with the Notepad was fun, too. That was until sophisticated Content Management Systems like Drupal and WordPress came into being. The same thing is getting ready to happen to REST APIs, and it’s going to change enterprise software development forever.

The Client-Server Interface Negotiation

I want to spend a minute looking at how we got here. To make a very long story short, mobile applications and the API economy crept up on enterprise developers and created a massive paradigm shift. Mobile networks are replacing Ethernet, JSON is replacing HTML, and RESTful services are replacing application servers. The client-server pendulum has swung back to the smart client. Applications are becoming smarter, more atomistic, and more widely distributed. The Internet of Things and the distributed services network will accelerate this trend. Others have written extensively about this big shift to the client.

Slide1.pngThere is an aspect of this situation that is poorly understood, but it gets to the fundamental heart of the problem. In the old world, creating an enterprise application meant creating a folder full of files on the server. Then smart client applications and modern developers hit the scene. Now there are two folders full of files: one on the server, and one on the client. See what happened? The complexity doubled.

Or maybe tripled, because now there is a third thing that is even worse: the interface negotiation. Now the server team and the client team have to agree on an API. And these are very different groups of people. Modern client developers are interested in user experience, art direction, and application design. The server team is interested in administration, infrastructure and security. They probably can’t agree on where to have lunch, much less on how to build an API.

Slide2.png
The REST API Complexity Problem

But if the client and the server teams can get their act together, they will be able to create a REST API for the project at hand. Perhaps they are using a powerful API creation tool. Strangely enough, this is where the real damage starts to happen.

The modern enterprise may need hundreds or even thousands of mobile applications. And so they build more and more REST APIs, over and over again, for each new application, with different developers and frameworks, on different pieces of infrastructure, until they end up with a serious backend complexity problem that is impossible to scale, port, or secure.

Slide3.png
And by the way, they have to spend millions of dollars in development costs and hundreds of man-years in hard labor for the honor of achieving these terrible results. In many situations, the API services they build are not fully documented. I have talked with customers that were not even able to identify all of their endpoints. And there is no way that they are going to throw it all away and start work on a general purpose API platform, because these applications are in production.

Building new REST APIs for every new project is a really bad idea, and it leads to an avalanche of other bad ideas and expensive mistakes. Remember that fancy tool that helped you build the APIs quickly? It really just helped you hit the complexity wall faster. Are you losing track of all your services? Maybe API Management tools will help. Problem is, they increase overall complexity and also require an additional proxy relay. Maybe we can secure this mess from the client with Mobile Device Management. By the way, we need to buy API testing tools to make sure all of these special case services are working correctly.

API Automation To The Rescue

API Automation is the solution to this problem. Don’t think about the APIs you need for the project at hand. Focus instead on the backend data sources that need to be mobilized. Then build a general purpose and reusable REST API Platform capable of supporting long-tail software development. This strategy changes everything. The client and server teams are decoupled. The server guys administer the platform, the client guys build the app. The number of external endpoints is minimized. All of the services are documented automatically.

An API Platform strategy also delivers enormous value to customers. Client developers instantly receive all of the services they need. They can use the same APIs for multiple projects. The backend infrastructure becomes more scalable, portable, reliable, and secure. The total number of external endpoints and services is consolidated. For the organization as a whole, API Automation increases standardization, compliance, and governance.

But there is one little problem with this advice. Because there is no way to know HOW to create a general purpose REST API Platform until AFTER you have built a whole bunch of applications on top of it. You have to build it and then throw it away quite a few times before you figure out how to do it correctly. This is exactly the type of thing that the modern enterprise never does. Most often they just build new APIs for every project until the entire backend system grinds to a halt.

Slide4.png
What we need is a tireless group of domain experts to create a reusable API Platform and release it as an open source project under the Apache License so that enterprise companies everywhere can adopt a platform strategy. In fact, the project should be easy to install on any cloud or server, so that there is no API lock in, either. Then companies could migrate apps from development to test to production or between the cloud and data center with ease. If you would like to learn more about just such a platform, please check out our website at www.dreamfactory.com.