The Reusable REST API Platform Strategy

The engineering team at DreamFactory designed and built some of the very first applications that use web services. Over the years, we learned many lessons trying to create the perfect mobile backend for these applications. This article lays out some of the problems companies encounter when they build custom REST APIs from scratch, and after that, we look at some of the architectural advantages and technical characteristics of a reusable REST API platform.

Table of Contents

• The REST API Complexity Problem
• Moving to a Reusable REST API Strategy
• Characteristics of a Reusable API
• Conclusion

The REST API Complexity Problem

When a company decides to start a new mobile project, the IT group first defines the business requirements, and then starts writing the actual software. Usually there is a client-side team that designs the application and a server-side team that builds the backend infrastructure.

These two teams must work together to develop a REST API that connects the backend data sources to the client application. One of the most time-consuming and expensive aspects of the development process is the “interface negotiation” that occurs between these two groups.

With the acceptance of BYOD and general proliferation of mobile devices, the modern enterprise may need hundreds or even thousands of mobile applications. New mobile projects typically have new requirements that were not anticipated by the existing services. You could go back and try to expand the scope of the old services, but they are already in production, and so in many situations a new REST API is created for each new project.

And so the API building process continues over time with various developers, consultants, and contractors. The REST APIs are often written with different tools and developer frameworks. They run on different servers or in the cloud. REST APIs are tied to different databases and file storage systems. Each new service has different security mechanisms, credential strategies, user management systems, and API parameter names, as this diagram illustrates.

The result, as you can imagine, is a giant mess. Data access rights are confused, user management is complex, and application deployment is cumbersome. The system is difficult to manage, impossible to scale, and full of security holes. API documentation is nonexistent. Often, companies can’t define what all the services do, or even where all of the endpoints are located.

Moving to a Reusable REST API Strategy

The mistake here is that development activity started with business requirements and application design, and then worked back to server-side data sources and software development. This is the wrong direction. It’s better to first identify the data sources that need to be mobilized and then create a comprehensive and reusable REST API platform that supports general-purpose application development.

There are many benefits to adopting a reusable REST API strategy. There is no need to keep building server-side software for each new project. Client-side application design is decoupled from security and administration. The “interface negotiation” is simplified. Development expenses and time to market are dramatically reduced. Developers don’t have to learn a different API for each project. RESTful services are no longer tied to specific pieces of infrastructure. Applications can be moved between clouds or from testing to production with ease.

At this point some of you might be wondering what the technical characteristics of reusable REST APIs are, and how they should actually be implemented. Unfortunately you can’t learn this until you have built many RESTful APIs incorrectly, and at that point it’s usually too late. We have been working on this problem at DreamFactory Software for 10 years now, and we had to start over many times before hitting on the right design pattern to enable our developers to build applications out of general-purpose interfaces.

Characteristics of a Reusable API

There are some basic characteristics that any reusable REST API needs to have. The API needs to support both HTML5 and Native Mobile access patterns. Requests and responses should include JSON or XML with objects, arrays, and sub-arrays. Noun based endpoints should be automatically generated depending on the database schema. All HTTP verbs (GET, PUT, DELETE, etc.) need to be implemented for every use case. Support for web standards like OAuth, CORS, GZIP, and SSL is also important.

There needs to be a consistent URL structure for accessing any backend data source. The File Storage API should be a subset of the NoSQL API, which should be a subset of the SQL API. Parameter names should be reused across services where possible. This presents developers with a familiar interface for any data source. The API should include interactive documentation that allows developers to quickly experiment with different parameters.

In general, the structure of the request URL and associated parameters needs to be very flexible and easy to use, but also comprehensive in scope. Looking at the example below, there is a base server, an API version, the backend database, and a particular table name in the request URL string. Then the parameters specify a complex filter with a field name, operator, and value. Lastly an additional parameter sorts the returned JSON data array.

A huge number of application development scenarios can be implemented just with the filter parameter. This allows any subset of data to be identified and operated on. For example, objects in a particular date range could be loaded into a calendar interface with a filter string. Complex logical operations should also be supported. The filter string interface needs to protect against SQL Injection attacks. Other database specific features include:

• Pagination and sorting
• Complex logical operations
• Rollback and commit
• Role based access controls
• Record level access controls
• Stored functions and procedures

The DreamFactory API platform supports operations on arrays of objects, but you can also specify related objects as a URL parameter. This allows complex documents to be downloaded from a SQL database and used immediately as a JSON object. The data can be edited along with the related objects. When committed back to the platform, all of the changes are updated including parent, child, and junction relationships. This flexibility supports a huge number of very efficient data access patterns.

The vast majority of application development use cases can be supported with a reusable REST API right out of the box. For special cases, a server-side scripting capability can be used to customize existing services or create entirely new ones. DreamFactory uses the V8 JavaScript engine for this purpose. Here are some of the special cases that you might want to implement with server-side scripting:

• Custom business logic
• Workflow triggers
• Formula fields
• Field validation
• Web service orchestration

Conclusion

REST API complexity is an important problem for companies building mobile applications. The tendency to build new APIs for each new project has negative consequences over time. Adopting a REST API platform strategy with reusable and general-purpose services addresses this problem and provides many benefits in terms of more agile development and quicker time to value. In this paper I have tried to provide some of the lessons we learned about constructing reusable REST APIs while developing the DreamFactory platform — I hope this provides a useful template as your enterprise embarks on mobile application development.

Related reading: API Integration - What Is It?