How We Got Here | Dreamfactory

Back before the turn of the century people started experimenting with new ways to make function calls across the firewall. Around this time in 1998 Dave Winer invented XML-RPC, which stands for Extensible Markup Language Remote Procedure Call. His specification introduced the concept of exchanging XML documents across a URL endpoint. The request and response would constitute a function call. These XML documents were normally exchanged with an HTTP POST transaction, and since this looked like regular web traffic the function calls sliced right through most firewalls. Tony Hong stated the XMethods website about this time, and soon there were many example services that could be called.

For my part, I was a rich client pioneer, I wrote World Builder, Course Builder, and Supercard, among other applications, all focused on building better user experiences. One weak point was that there was no consistent way to hook these front-end applications up to back-end data. But the new document exchange technologies promised to change all that: here was finally a standard way to hook up to any data source that supported the protocol and conduct transactions using the Internet. So I set about building a rich client front-end for the emerging service-based back ends. The vision was that rich applications could be cobbled together from services from all over the world. The first version of the DreamFactory Player was written around this time. The DreamFactory Player is an ActiveX Control, Netscape Plugin, or stand-alone executable that runs on Windows and Macintosh.

Shortly thereafter Dave, along with Don Box at Microsoft and a team from IBM, started working on the SOAP specification, which stands for Simple Object Access Protocol. This changed things a little bit. With XML-RPC the idea was that a complex function could return an answer. With SOAP both the request and response were complex XML documents that were exchanged across a fixed URL endpoint. The SOAP specification evolved into a very ambitious attempt to encode communications of many types, and then the WSDL specification further complicated this standard. Around 2001 Marc Lucovsky released Microsoft's HailStorm product which used SOAP to expose a complex data set for storing information. We built some very interesting apps on top of HailStorm, including an amazing animated collaborative calendar, but Microsoft killed this project a year later. In retrospect, HailStorm was way ahead of its time.

At any rate, exciting new services kept being developed. Back in 2000, Roy Fielding's dissertation introduced REST, which stands for Representational State Protocol, and we added REST support to the DreamFactory player. Services that used REST started to appear around 2003. With REST you could largely dispose of the request document. The URL itself simply specified the resource that was requested. The return document was structured XML with the information. There was no required format for the document, and shortly thereafter people also started using JSON as well. The JSON format, which stands for JavaScript Object Notation, had the advantage of being the native format for JavaScript arrays. A JavaScript program could simply get the document and use it without translation.

Working on applications with the DreamFactory Player, we quickly learned that we needed an orchestrated palette of services to build something compelling. For example, with a traffic service you could send a request with a California Highway number and receive a description of the traffic conditions at that time. While this might be a cool thing to do, there is no easy way to build a compelling application out of that service. Around 2003, salesforce.com introduced an XML-RPC interface to their Customer Relationship Management database, and shortly thereafter they introduced a SOAP interface. This was what we needed: a complete services palette with pre-built objects. Salesforce also had an HTML interface that we could use for single sign-on and embedding the DreamFactory Player.

When salesforce launched the AppExchange in 2005, we wrote the first third-party application, our DreamTeam Project Manager. We started to acquire new enterprise customers, and were funded in 2006 by one of the premier venture firms specializing in enterprise software, New Enterprise Associates. Over the next 5 years we published more than a dozen applications on salesforce, and also started working with other cloud platforms, including Intuit Partner Platform, Cisco Webex Connect, Amazon Web Services and Microsoft Azure.

Our engineering team became extremely knowledgeable about what makes a good services platform, what the problems were at the client, and which ones worked the best. We also started building our own service platforms on generic cloud infrastructure for specific projects. And today we have 15,000 companies using our rich application products around the world. Along the way, we learned some very interesting things.

First, there are a huge number of different applications that can be built if you have the right palette of services. By publishing rich applications on lots of different service platforms, we eventually discovered a very comprehensive and general purpose interface that supported generic application development. These service calls can be combined like LEGOs to build darn near anything.

Second, our new development model was incredibly efficient. Once you have the right set of services, the server-side is baked. After that a small team or even a single developer can build and deploy an application without much support. You don't need to write server-side software anymore, and the server team and the client team don't have to discuss integration issues. This dramatically changes the time and cost considerations for building and maintaining an application.

Third, the rich client model has huge relevance for the new world of mobile devices. Mobile networks are often quite slow and they can lose connection altogether when moving between zones. Rich client applications minimize network usage because only the data is shipped back and forth. And since the data transactions are atomistic and under client control, this model supports occasionally connected applications as well.

Lastly, this model also saves money on the cloud. When business logic is moved to the client, the server becomes less complex and more agile, and this reduces the cost of hosting cloud infrastructure. Server-side page generation sends graphics, layout, scripting, and data across the wire with each user action or new screen. Moving to a service-oriented architecture can improve network performance by 90% or more, and this lowers the cost of mobile data plans.

And so a few years ago, our engineering team started work on the DreamFactory Services Platform in an effort to bring these insights to a wider community of developers. Here are the design goals we set for ourselves:

We knew that enterprise customers would need the ability to install our open source platform on their own cloud or data center. This way they could manage infrastructure, cost and security with familiar tools.

We wanted to leverage our years of experience using service platforms and put together a comprehensive palette of services that was versatile, easy to use, and very powerful. This needed to include a direct connection to a dedicated SQL database.

We put special emphasis on HTML5 support with native JSON documents. We think HTML5 is emerging as the right choice for enterprise applications. Companies need to support any device, and they don’t really need the app store for distribution.

We wrote an Admin Console to make platform management a breeze. Our Admin Console is an HTML5 application written in Angular JS that simply calls our own services.

And that’s how we got here. The first version of the DreamFactory Services Platform will be available on April 30th, 2013. Where we go next is up to you.