API Calculator: Understanding the costs behind building an API-based application

Do you want to save money, deploy projects faster, and spend more time on developing application experiences that enthrall users? Then it’s practically a certainty APIs will play a critical role in your software development process. Accordingly, you’ll want to obtain a fairly accurate understanding of the cost and time required to building an API. But it’s important to understand that there’s much more to an API than just coding an interface to some data source such as a database. Whether you are planning on hiring a contractor or assign a new project to your team, an API calculator can help you understand the time and cost required to develop a mission-critical part of your next project. Continue reading “API Calculator: Understanding the costs behind building an API-based application”

Improved Data Security with MySQL Privileges and DreamFactory

DreamFactory and MySQL

All MySQL installations naturally include a root account and offer the ability to create restricted user accounts. However, otherwise sane developers will often use these root accounts for application-level communication, dramatically raising the likelihood of data theft, data exfiltration, and other security issues. For that reason the DreamFactory team always recommends users take care to create restricted MySQL users before using the platform to generate APIs.

In this tutorial, you’ll learn how to create a non-root MySQL user and then further restrict this user’s privileges to a specific database and even table subset. You’ll also learn how to subsequently revoke a user’s privileges to reflect changing requirements.

Continue reading “Improved Data Security with MySQL Privileges and DreamFactory”

SOAP vs. REST APIs – Which Reigns Supreme?

These days it’s more true than ever that “no company is an island”. Many businesses rely on each other by exchanging information and much of that exchange is done via an API. When diving deeper into the question of developing APIs, you’ll undoubtedly encounter the question: SOAP or REST? Although REST APIs have become the most popular choice for today’s businesses, the decision isn’t always an easy one. In this article, we’ll go over everything you need to know about SOAP and REST APIs. That way you can come to the conclusion that’s ultimately right for your situation.

What is an API?

There are a lot of definitions of the term “API” out there, and many can be confusing. An API (Application Program Interface) is a way for you to get information needed in a consistent format. You can think of an API as like an interaction between a business and a customer, such as placing an order at a restaurant.
  • Customers first read the menu, then they decide what food they would like to order.
  • The waiter serves as the “middleman” between the customer and the business. They take the request and present it to the business in a way that’s most comprehensible and efficient.
  • The business reviews the request and sends back a response to the customer, such as a plate of food.
It’s important to note that in this example, the interaction is entirely predictable. When customers go to a restaurant, they can assume that they’ll be handed a menu. Then use that menu to place an order, and receive the food they ordered. In the same way, APIs offer consistency to users who want to query a website for its data. By establishing a common set of rules for exchanging information, APIs make it easier for two parties to communicate.

What is a SOAP API?

SOAP (Simple Object Access Protocol) is an API protocol that uses the XML Information Set specification in order to exchange information.


A standard SOAP message consists of the following XML elements:
  • An Envelope element that identifies the document as a valid SOAP message
  • An optional Header element that specifies additional requirements for the message, such as authentication.
  • A Body element that contains the details of the request or response.
  • An optional Fault element that contains information about any errors encountered during the API request and response.

What is a REST API?

REST (Representational State Transfer) is an architectural style for APIs that relies on the HTTP protocol and JSON data format to send and receive messages. REST utilizes CRUD (Create, Retrieve, Update, and Delete) to keep API calls as simple as possible to understand. With its simplicity, developers can understand it whether they are the lead architect or a junior developer just starting.


As highlighted by BBVA some major advantages to REST Visibility, reliability and scalability. REST is very simple to scale and can be reused across several projects. This provides you with the advantage of faster development cycles with far less headaches. It also is independent from being bound to a single language, meaning you can use it with any language such as PHP, JavaScript, or Python.

The Pros and Cons of SOAP and REST

When comparing REST and SOAP, people often use the analogy of a postcard and an envelope. REST is like a postcard in that it’s lightweight and consumes less bandwidth (paper). Meanwhile, SOAP is like an envelope: there’s an extra overhead required on both ends to package and un-package it. Note that the analogy isn’t perfect: unlike a postcard, the content of REST requests and responses isn’t (necessarily) insecure. Instead, REST uses the security of the underlying transport mechanism, which is usually HTTPS. On the other hand, SOAP implements its own security measure, which is known as WS-Security. Some people believe that REST is largely a “replacement” for SOAP, due to its lower overhead and improved ease of use. According to Cloud Elements’ 2017 State of API Integration report, 83 percent of APIs now use REST, while only 15 percent continue to use SOAP. Some of these businesses primarily use REST, but continue to integrate SOAP APIs into their projects using tools such as DreamFactory’s SOAP connector. However, the thought of SOAP being outdated isn’t quite accurate. Even as REST becomes the API style of choice for most businesses, SOAP remains a tool that is better-suited for certain use cases, mainly in large enterprises who need additional extensibility and logic features native to the protocol.

The advantages of REST include:

  • Flexibility: Although REST is most commonly implemented with HTTP and JSON, developers are by no means obligated to use them. Websites can send back responses using data formats including JSON, XML, HTML, or even plaintext-whatever best suits their needs.
  • Speed: Because it tends to use much less overhead, REST APIs are typically significantly faster than SOAP. While the differences might be imperceptible for a single request, the disparity grows larger and larger as you place more and more requests.
  • Popularity: REST has reached critical mass on the Internet. Major websites such as Google, Twitter, and YouTube all use REST APIs for users to send and receive messages. Due to this familiarity, it’s typically easier for developers to get up and running with REST.
  • Scalability: Thanks to their speed and simplicity, REST APIs usually perform very well at scale.
Despite the major benefits of using REST, SOAP remains the preferred protocol in certain use cases. Some organizations find that SOAP offers the transactional reliability that they’re looking for, while others simply continue to use SOAP because they need legacy system support.

The advantages of SOAP include:

  • Formality: SOAP can use WSDL (Web Services Description Language) to enforce the use of formal contracts between the user and the website. SOAP is also inherently compliant with ACID database standards, which ensures that the transactions it performs will be valid even in the event of errors or hardware issues.
  • Logic: If a REST API request is unsuccessful, it can only be addressed by retrying until the request successfully goes through. On the other hand, SOAP includes built-in successful/retry logic so that the requesting system knows how to behave.
  • Security: SOAP comes with its own security mechanism, WS-Security, built into the protocol. If you want to ensure that your messages are secure, rather than relying on the underlying transport mechanism as does REST, then SOAP may be the right choice.
  • Extensibility: In addition to WS-Security, SOAP includes support for other protocols such as WS-Addressing and WS-ReliableMessaging that can define other standards of communication and information exchange.

Converting SOAP to REST


DreamFactory can convert your SOAP service to REST in minutes. If you have your SOAP service RESTified you can take advantage of the features DreamFactory provides. Like turning security up 10 notches by securing the API with API Keys and Roles to only allow access to people that have the right to it. Even add custom events and business logic by way of our powerful Scripting engine.


For most cases, REST should be considered the “default” option as adoption continues to grow across the web. Most public-facing APIs now use REST, because it consumes less bandwidth and its compatibility with HTTP makes it easier for web browsers to use. However, you may find that the additional features and security offered by SOAP are enough to sway your decision. In the end, the “right” choice between SOAP and REST will be highly dependent on your own situation. Even better, the choice of SOAP and REST doesn’t have to be between one and the other. If you want to communicate with REST but still need access to legacy SOAP services, DreamFactory offers the ability to add a REST API onto any database or SOAP API. See how quick and easy it is!

If you have your SOAP service RESTified we can take advantage of the features DreamFactory provides. Like turning security up 10 notches by securing the API with API Keys and Roles to only allow access to people that have the right to it. Even add custom events and business logic by way of our powerful Scripting engine. If you know REST APIs are the best for you, don’t waste any more time building them. Try it yourself with your own instance of DreamFactory or connect with an API specialist today!

Changing An API Key For One of Your Apps In DreamFactory

So what happens if you make a mistake and expose your admin app api_key or just need to change api_key associated with one of your apps?  We have an easy workaround that doesn’t require you to have to change any of your endpoints or having to recreate an app, etc.  This article shows you how to access all of your app API keys via MySQL or, if you haven’t fully started exploring DreamFactory yet, the default SQLite database.

Continue reading “Changing An API Key For One of Your Apps In DreamFactory”

How To Configure An ELK Stack With DreamFactory

DreamFactory has had support for Logstash since version 2.3 for our Gold Tier version.  Elastic makes some great tools to support very robust logging.  Incorporating Elasticsearch, Logstash and Kibana into your powerful, scalable DreamFactory instance is a no brainer, especially for users who have a lot of data being pushed and pulled through various endpoints.  This will make the lives of your admins so much easier with the amount of detail they can grab to troubleshoot issues. Continue reading “How To Configure An ELK Stack With DreamFactory”

Automatically build APIs from your Oracle database using DreamFactory

DreamFactory will save you and your company hundreds of hours by auto-generating an API automatically for every major SQL database, including Oracle. In this tutorial, we’re going to walk through the process of connecting DreamFactory to your Oracle database, review the features automatically built into DreamFactory that’ll help you manage and customize your API, and go over scripting support and security features. Continue reading “Automatically build APIs from your Oracle database using DreamFactory”

NoSQL, No Problem!

LeeHicksYou may have already seen that DreamFactory Service Platform (DSP) supports SQL databases with its REST API, but did you know that we also support most of the popular NoSQL databases? Currently, DSPs support Amazon Web Services’ SimpleDB and DynamoDB, Windows Azure’s Tables, MongoDB, MongoHQ, and CouchDB.

While the technology behind, and the interface in front of, each of these services vary, some dramatically, they each support storing data and lots of it. Some already have a REST or somewhat  REST-like HTTP interface to access them, which may mean storing your access keys or authorization information on the client and passing them along with every call. Some do not have an HTTP interface at all and must be accessed through server-side code that you have to go find and install or write yourself. Some allow you to design the primary keys or indexes of your storage tables, some do not. Some support JSON natively, some do not. In fact, the formats supported for data entry and retrieval (think filtering) vary almost as widely as the number of vendors. Get the drift? Different creators, different creations. But you say “I just want a simple way for my app to get my data in and out, quickly and easily.” That’s where I think we can help.

NoSQL as a Service

DSP Services allow you to “on-board” web-based or locally running services that you want to use from your app and access from one authenticated REST API. The access to these services can be controlled by app and by user role. There is more on this and how to provision these services in our documentation here.

The same goes for the supported NoSQL services. We securely store any credentials required to access your storage on the server-side, relieving your app from maintaining it. We remove any cross-site hoops your app would have to handle by making it a “native” REST service on the DSP. We even go a step further and provide a common, fully REST-based API to each of the supported NoSQL service types and use a common data format layout, using JSON, for data entry and retrieval. Where we were able, we also provide a common SQL-like filtering language, making it dirt simple to find the data you need.

We like to call this our “blend layer”. The idea is, just like a DAO (data access object), we simplify the process to use your data. In fact, we have made this so “blended” it is nearly identical to our SQL DB REST API, which we think is powerful but simple to use. So how does this help you? Well, you write the app once, and store your data in your favorite storage. If or when your “favorite” storage changes (i.e. if something cheaper or faster comes along), your app and data will only require minor changes (if any) to keep chugging along.

So what does this wonderful interface look like? Well, since you asked so nice and all. Once you get your service setup on your DSP (using the documentation mentioned earlier, or the vendor-specific follow-up blogs to this blog), you can use our Live API documentation interface to see how it works. Of course, you can use cURL or your favorite REST testing browser app, it is up to you. Since I am such the promotionalist, I’ll use the Swagger interface in this blog to show you.

As with the other DSP services, we put a mostly standard meaning to the HTTP verbs in our REST API. Here is the how they are used for NoSQL services (using “mongo” as our demo service).

DreamFactory MongoDB Rest API

Note that the ‘POST’ is used for creating new entities, ‘PUT’ is used for replacing the whole content of an entity and ‘PATCH’ is used to merge in changes to an entity (‘PATCH’ and ‘MERGE’ HTTP verbs are used interchangeably in the REST API). Also, as with the other DSP services, if your transport layer only accepts ‘GET’ and ‘POST’ verbs, using a ‘POST’ command while setting either a url parameter, ‘method=PATCH’, or a header, X-HTTP-Method = PATCH will “tunnel” the command through properly.

Table Administration

Azure calls them “tables”. So does AWS, that is for DynamoDb, but for SimpleDb, they are called “domains”. For CouchDb, they are called “databases”, not to be confused with MongoDb’s “databases” which house what they call “collections”. See what I mean?? While each vendor calls them something different, we will refer to them collectively as “tables”. These are the things that group your sets of data together or partition them, however you see it. You can actually manage most aspects of the table administration through the REST API.

To get a list of currently available tables, just send a GET request to the service’s root, which, if you are using one of our free-trial DSPs, looks like this.


Every DSP service, when queried at its root with no extra url parameters, returns an array of resources available by that service. In the SQL and NoSQL case, these resources are table names.

   "resource": [
         "name": "test"
         "name": "zipcodes"

To include vendor-specific table information, if any is available, you can send ‘include_properties=true’ as a url parameter in the request. You can also limit the returned data by specifying the tables by name, using the ‘names=table1,table2,table3’ url parameter. The returned data will include a table array, where each object in the array includes at least the ‘name’ field and value, as well as, any available vendor-specific properties of the table, i.e. keys, etc.

   "table": [
           "name": "test",
           "indexes": [
                   "v": 1,
                    "key": {
                       "_id": 1
                   "ns": "test.test",
                   "name": "_id_"
           "name": "uploads",
           "indexes": []

Use this same data format for creating new tables, or updating and deleting existing tables. You can also use the ‘names’ url parameter in a delete request to delete existing tables.

Record Administration

Again, whether they are called “entities”, “documents”, or “items”, we collectively call them “records”. These are the groups of name-value pairs that make up your app’s data. The DSP REST API provides the following ways for your app to retrieve, create, update and delete data from the tables.

Retrieving Records

When it comes to making your NoSQL data available to your app, our REST API provides several different methods to get just the data sets you want, when you want it. Here are the available options for a GET request on a particular table.

table get request

The table name is sent as part of the url, while the other options are sent as url parameters. If url parameters are not your cup of tea, you can send most of them as posted data (using POST request with X-HTTP-METHOD = GET as mentioned earlier. Note that the “record” parameter can only be sent in this way.). Most of them are self explanatory, but a few may need clarification.

  • ids” – This is a comma delimited list of unique identifiers (think primary key) values for retrieving multiple records at a time. If passed as POST data, this can also be an array of ids. In the case where they is no static identifier field (DynamoDb), use  of the “id_field” parameter is necessary to indicate which field these values belong to.
  • filter” – This is where you use our simple SQL-like filter string. Comparison operators supported on every platform are ‘ = ‘, ‘ != ‘, ‘ > ‘, ‘ >= ‘, ‘ < ‘, ‘ <= ‘; or as their SQL short form ‘ eq ‘, ‘ ne ‘, ‘ gt ‘, ‘ ge ‘, ‘ lt ‘, ‘ le ‘. Depending on the db vendor, others such as ‘ contains ‘, ‘ like ‘, and ‘ begins_with ‘. Spaces surrounding the operator is required. String values on the right side of the comparison must be within single of double quotes. Depending on the db vendor, logical comparisons (AND, OR, NOT) are also supported. More on the differences in later blogs. The whole filter string must be url-encoded.
  • record” – Used only in POST data, this option allows you to send partial (minimally the identifier fields) or complete records back to the database to be “refreshed” with the latest values. This is also helpful in instances where the table has multiple indexes, i.e. DynamoDb and Azure Tables.

For example, if we want to find the first 3 records in the zipcodes table information that have a population of over 20,000 people, returning only the city, state and percentage over 40 years of age. If you are familiar with SQL, this would look like “SELECT city,state,PopulationOver40 from zipcodes WHERE pop > 20000;”. Here is what it looks like in our REST API.


And this is what gets returned…

 "record": [
     "_id": "28659",
     "city": "NORTH WILKESBORO",
     "state": "NC",
     "PopulationOver40": 9.19
     "_id": "31201",
     "city": "HUBER",
     "state": "GA",
     "PopulationOver40": 99.877
     "_id": "71291",
     "city": "WEST MONROE",
     "state": "LA",
     "PopulationOver40": 80.099

Creating Records

Use this same data format above for creating new records. One could post an array of records, or a single record like this…

POST https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes
     "_id": "29684",
     "city": "STARR",
     "state": "SC",
     "pop": 2889,
     "PopulationOver40": 82.4

If the DB vendor does not automatically create the primary key, it must be included in the POST request. The applicable keys are always returned for successfully created records on a create request.

Updating and Merging Into Records

The same array of records or a single record format with changes is supported for updating records. Using the PUT HTTP verb as mentioned above will replace the whole record with the posted data if found by matching identifiers in the record.

PUT https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes
     "_id": "29684",
     "city": "STARR",
     "state": "SC",
     "pop": 2890,
     "PopulationOver40": 79.4

If you only want to merge changes into a record without having to reset everything, then use the MERGE or PATCH HTTP verb and send only the changes along with the identification fields.

PATCH https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes
     "_id": "29684",
     "pop": 2890,
     "PopulationOver40": 79.4

If you only want to update or merge data for one record, and it can be identified by a single key field, then you could also add the id to the end of the url, and pass only the fields that require change.

PATCH https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes/29684
     "pop": 2890,
     "PopulationOver40": 79.4

Two other methods of merging data are by id list or filter (using “ids” or “filter” url parameters mentioned above for retrieving records). In these cases, if not natively supported, the server will query the table for the filtering results, merge in the record changes and push the changes back in an update request. This is an easy way of updating multiple records with the same field-value changes.

Deleting Records 

Deleting is similar to updating records but no posted data is required.

More to Come…

Stay tuned for a series of blogs on each of the supported NoSQL services and how to use them in your next awesome app.

Adding a Remote Web Service to Your DSP

ToddAppletonThe DreamFactory Services Platform (DSP) provides a flexible and powerful backend for your mobile apps. If you don’t have a DreamFactory account, sign up here. One of the key features is the ability to integrate any number of remote web services, and access them directly through the DreamFactory REST API. In this blog post we’ll walk through the process of setting up and invoking a remote web service. Continue reading “Adding a Remote Web Service to Your DSP”