Using OAuth with the New Portal Service

As all developers know, your creations can become tired, bored, and sometimes angry when forced to talk to only a single server or service. Applications are, for all intents and purposes, tiny monsters: they grow hungry to communicate with others to expand their knowledge, reach, and power. We at DreamFactory know this all too well and wanted to create an easy way to allow your monsters to feed their yearning. With the next release of the DreamFactory Services Platform™ (version 1.1.3), satisfying that need will become a whole lot easier. What makes this possible you ask? A new system service called Portal does. Continue reading “Using OAuth with the New Portal Service”

The New Mobile Architecture

There are a number of software systems for generating HTML pages on the server side, including Websphere, Weblogic, JSP, PHP, Python, and Ruby.

But the increasing prevalence of mobile devices has started to chip away at the usefulness of this strategy. The “click, get a page” model works well enough on a desktop browser with a dedicated connection, but on a tablet or phone this is a cumbersome and heavyweight solution. Each page is likely to carry graphics, page layout, scripts, and other elements along with the actual data. This is a lot of content to download for just another screen update, especially with an expensive data plan. Intermittent connectivity is also an issue as mobile applications move between zones or into areas without coverage altogether.

The rich client model has been around a long time, I’ve been working in this area since the late 1990’s, but the mobile revolution has brought this idea into the mainstream of application development. The rise of the native application model on iOS further demonstrated the advantages of being rich. The development of HTML5 was the browser’s way of supporting this model, and as the number of different mobile devices increase the popularity of this approach will grow, especially for enterprise applications.

This big change at the client suggests the need for some corresponding big changes on the server, but where are the ideal platforms for rich client development?

This is exactly the motivation for the DreamFactory Services Platform. A service platform can conduct compressed document exchange transactions with a mobile client and achieve very large reductions in network bandwidth.

Since the services are atomistic and under client control, occasionally connected applications are also possible. Because the interfaces are standards based, many different client environments are supported, including HTML5, native application, and server to server communication.

In the old model all the graphics, logic, and user interface must be generated on the server side. But when we move to a rich client model, there is an opportunity to replace all of that infrastructure with a pre-built and standards-based service platform. Based on our experience building rich client applications, we know that the right palette of services will support a very large number of application scenarios.

Since the services interface talks directly to the database, the speed and performance of the system is maximized. And because there is no need for user experience or graphics generation on the server, this model is much less expensive to deploy on a pay per use cloud server. But the most significant cost savings are in time to market and reduced software development. More about that in the next post.

How Circa5 Connects an iOS Mobile App to DreamFactory

BenBusseThis case study provides an overview of how Circa5 uses DreamFactory as a backend for Heisenbug, a collaborative bug tracking system designed for mobile devices. The case study describes how the iOS version of the application interacts with the DreamFactory open source backend services platform.

Heisenbug iPhone resized 600

We refer to each instance of the DreamFactory platform as a “DSP” which is short for DreamFactory Services Platform. As you’ll see, each DSP provides a database and a full-featured REST API to simplify the process of building your mobile application.

App Overview

Heisenbug is a simple issue tracking system that allows users to log and track project issues from their phone or tablet. The app is designed to motivate and encourage resolution using a gamification points system.

The primary use case for Heisenbug is creating and tracking “projects.” A project is either a bug to fix or a feature to complete. The details of a project are configurable to suit the working methods of each Heisenbug user. For example, a project can have an executive overview as well as more detailed information appropriate to an individual tasked with finishing the project.

Heisenbug motivates users to complete projects with a points system, a leveling game mechanic, and fun encouragement messages. For example, a user can close a bug to earn points and receive positive affirmation after reaching each level. The app provides a series of default phrases that can easily be customized.

The application is designed for collaboration. Groups of people can easily collaborate to finish a particular project. An access management screen allows an administrator to configure user permissions and sharing assignments for organizations, groups, or single-user accounts working on projects.

Other collaboration features include a Share feature that displays the ongoing discussion that a user is part of and as well as simple workflow notifications with SMS and email. These features help users stay in the loop when project information changes, enabling better collaboration across a project team.

Heisenbug iPad affirmation resized 600

Administering the DreamFactory Backend

Before building the frontend application, you need to configure your backend settings using the Dreamfactory admin console.

The first step of setting up the your backend instance is to create the application. On the left side menu choose Applications, enter your app name, check the box for native app, and click the “Create Application” button.

The next step is to create a Role for users of the application. Go to the Roles tab and enter a role name, for example “End User.” You need to associate the role with your application and also select which services are available for this role. Heisenbug end users all have the same service permissions, so the configuration is simple: just select “All” from the dropdown list and click “Add Service Access.” Don’t forget to check the “Active” checkbox before saving.

describe the image

Setting Up Database Tables

DreamFactory comes with a SQL database and a simple admin console for defining tables and fields. You can also import schema definitions in JSON format.

You set up your application’s schema from the “Manage Schema” tab. The Heisenbug schema consists of Records, Components, Messages, Priorities, Severities and Statuses. The tables are connected using SQL relationships. For example, the Records table references the Priorities table using a foreign key relationship. You can explore the complete schema using the Live API /schema.

Note: You can also easily connect RESTfully to your own SQL or NoSQL database by simply adding your database as a service in the DreamFactory admin console. This blog post provides an example of how to connect to your own database.

describe the image

Using the Live API Documentation

In the DSP admin console, you can click on the “API Documentation” tab to view every available API call. The API is documented with a tool called Swagger, which lets you quickly learn about each API call by viewing the request URL, response URL, and data returned in JSON format.

This is a powerful capability because it enables you to set up your backend, create some sample data, and make API calls before you even start implementing your application. For example, Heisenbug makes an API call to log users into the application to start a session. With Swagger, we could quickly see how the POST/user/session API call works before we implemented the call in our app.

Circa5 used an open source library called AFNetworking to make API calls from the client app. AFNetworking is very popular among iOS developers and is hosted on GitHub here.

describe the image

API Calls for User Authentication and Sessions

User management is a common requirement for many applications. DreamFactory’s REST API for user management made it very easy to manage user authentication, user roles, and session security.    

The Heisenbug app uses DreamFactory’s REST API for user authentication and session management.

When a new user registers, a POST is made to /user/register to create a new user in the database.

 -(void)
 registerWithEmail:(NSString*)email 
 firstName:(NSString*)firstName 
 lastName:(NSString*)lastName 
 displayName:(NSString*)displayName{
 
 static NSString *const baseURLString = 
 @"https://dsp-creativeappsagencydemo.cloud.dreamfactory.com";
 
 NSString *userSessionURLString = 
 [baseURLString stringByAppendingString:@"/rest/user/register"];
  
 NSDictionary *parametersDictionary = 
 [NSDictionary dictionaryWithObjectsAndKeys:email,@"email",firstName,@"first_name",lastName,@"last_name",displayName,@"display_name", nil];
 
 AFHTTPClient *httpClient = 
 [[AFHTTPClient alloc] 
 initWithBaseURL:[NSURL URLWithString:baseURLString]];
 [httpClient setDefaultHeader:@"X-DreamFactory-Application-Name" value:@"Heisenbug"];
 [httpClient setParameterEncoding:AFJSONParameterEncoding];
 
 NSMutableURLRequest *request = 
 [httpClient requestWithMethod:@"POST"
 path:userSessionURLString
 parameters:parametersDictionary];
 
 AFJSONRequestOperation *operation =
 [AFJSONRequestOperation 
 JSONRequestOperationWithRequest:request
 success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {NSDictionary *jsonDic = (NSDictionary *)JSON;
 
 //registration completed successfully
 }
 
 failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON) {NSLog(@"%@",error);
 
 //registration error                                                
                                                       
 }];
 
 [operation start];
   
 }

Each time a user logs into Heisenbug, a POST is made to /user/session to create a new session.

 -(void)
 loginWithEmail:(NSString *)email 
 password:(NSString *)password{
     
 static NSString *const baseURLString = 
 @"https://dsp-creativeappsagencydemo.cloud.dreamfactory.com";
 
 NSString *userSessionURLString = 
 [baseURLString stringByAppendingString:@"/rest/user/session"];
     
 NSDictionary *parametersDictionary = 
 [NSDictionary dictionaryWithObjectsAndKeys:email,@"email",password,@"password", nil];
     
 AFHTTPClient *httpClient = 
 [[AFHTTPClient alloc] 
 initWithBaseURL:[NSURL URLWithString:baseURLString]];
 [httpClient setDefaultHeader:@"X-DreamFactory-Application-Name" value:@"Heisenbug"];
 [httpClient setParameterEncoding:AFJSONParameterEncoding];
 
 NSMutableURLRequest *request = 
 [httpClient 
 requestWithMethod:@"POST"
 path:userSessionURLString
 parameters:parametersDictionary];
 
 AFJSONRequestOperation *operation =
 [AFJSONRequestOperation 
 JSONRequestOperationWithRequest:request
 success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {NSDictionary *jsonDic = (NSDictionary *)JSON;
 
 UserData *userData = [UserData sharedInstance];
 userData.sessionId = [jsonDic objectForKey:@"session_id"];
 userData.userId = [(NSNumber *)[jsonDic valueForKey:@"id"] intValue];
 userData.displayName = [jsonDic objectForKey:@"display_name"];
 userData.email = [jsonDic objectForKey:@"email"];
 
 User *user = [[User alloc] init];
 user.email = userData.email;
 user.userID = userData.userId;
 user.sessionId = userData.sessionId;
 user.displayName = userData.email;
 user.password = password;
                                                    
 [Utility saveUser:user];
                                                         
 [self.delegate loginResult:YES];
                                                        
 }
 
 failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON) {NSLog(@"%@",error);
 
 [self.delegate loginResult:NO];
                                            
 }];
 
 [operation start];
 
 }

API Calls for Database CRUD Operations

Creating, reading, updating, and deleting records is a universal requirement for any data-driven mobile application that stores information in a database. DreamFactory comes with a SQL database already installed and a straightforward REST API for CRUD operations, making it simple to operate on single records or multiple records at once.

Heisenbug uses a “bug” object to store information about bugs entered by users. In some cases, the application needs to perform CRUD operations on multiple records at once, for example fetch all the bugs assigned to a specific user.

In the case of multiple records, the application calls /db/{table_name} with either a POST (create), GET (retrieve), PUT (update), or DELETE (delete) API call. The DSP returns standard JSON in the API response.

For example, the following code performs a GET request to get a list of all bugs.

 -(void)requestBugs{
     
 static NSString *const DBBaseURLString = 
 @"https://dsp-creativeappsagencydemo.cloud.dreamfactory.com/rest/db/Bugs";
 
 AFHTTPClient *httpClient =
 [[AFHTTPClient alloc] 
 initWithBaseURL:[NSURL URLWithString:DBBaseURLString]];
 [httpClient setDefaultHeader:@"X-DreamFactory-Application-Name" value:@"Heisenbug"];
 [httpClient setDefaultHeader:@"X-DreamFactory-Session-Token" value:[UserData sharedInstance].sessionId];
 [httpClient setParameterEncoding:AFJSONParameterEncoding];
 
 NSMutableURLRequest *request = 
 [httpClient requestWithMethod:@"GET"
 path:DBBaseURLString
 parameters:nil];
 
 AFJSONRequestOperation *operation =
 [AFJSONRequestOperation JSONRequestOperationWithRequest:request
 success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {
  NSDictionary *jsonDic = (NSDictionary *)JSON;
  NSArray *records  = [jsonDic objectForKey:@"record"];
  NSMutableArray *bugs = [[NSMutableArray alloc] init];
                                                         
  [records enumerateObjectsUsingBlock:^(id obj,NSUInteger idx, BOOL *stop){
  
   Bug *bug = [[Bug alloc] init];
   bug.bugID  = [(NSNumber *)[obj valueForKey:@"id"] intValue];
   bug.title = [obj valueForKey:@"title"];
   bug.details = [obj valueForKey:@"details"];
   bug.points = [(NSNumber *)[obj valueForKey:@"points"] intValue];
   bug.rank =[(NSNumber *)[obj valueForKey:@"rank"] intValue];
   int type = [(NSNumber *)[obj valueForKey:@"type"] intValue];
   bug.type = (type == 1)? bugTypeBug : bugTypeFeature;
   bug.speedToComplete = [(NSNumber *)[obj valueForKey:@"speedToComplete"] boolValue];
   bug.milestone = [(NSNumber *)[obj valueForKey:@"milestone"] boolValue];                                                            
   bug.archived = [(NSNumber *)[obj valueForKey:@"archived"] boolValue];
   bug.severity = [(NSNumber *)[obj valueForKey:@"severity"] intValue];
   bug.priority = [(NSNumber *)[obj valueForKey:@"priority"] intValue];
   bug.status = [(NSNumber *)[obj valueForKey:@"status"] intValue];
   bug.component = [(NSNumber *)[obj valueForKey:@"component"] intValue];
   bug.assignedUser = [(NSNumber *)[obj valueForKey:@"assignedUser"] intValue];;
   bug.issueType = [(NSNumber *)[obj valueForKey:@"issueType"] intValue];
   bug.notes = [obj valueForKey:@"notes"];
 
  }];
 
  self.bugs = bugs;
  [self.delegate bugsUpdateResult:YES dataReturned:bugs error:nil];
 }
 
 failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON) {
  NSLog(@"%@",error);
  [self.delegate bugsUpdateResult:NO dataReturned:nil error:error];
                                                     
  }];
 
 [operation start];
 
 }

In other cases, the application simply needs to perform a CRUD operation on a single record, for example delete a specific bug.

In the case of single records, the application simply calls /db/{table_name}/{id} with the relevant HTTP verb for that record: either GET, POST, PUT, or DELETE. The DSP returns standard JSON in the API response.

 -(void)
 deleteBug:(Bug*)bug {
 
 NSString *DBBaseURLString = 
 [NSString stringWithFormat:@"https://dsp-creativeappsagencydemo.cloud.dreamfactory.com/rest/db/Bugs?ids=%d",bug.bugID];
  
 AFHTTPClient *httpClient = 
 [[AFHTTPClient alloc] 
 initWithBaseURL:[NSURL URLWithString:DBBaseURLString]];
 [httpClient setDefaultHeader:@"X-DreamFactory-Application-Name" value:@"Heisenbug"];
 [httpClient setDefaultHeader:@"X-DreamFactory-Session-Token" value:[UserData sharedInstance].sessionId];
 [httpClient setParameterEncoding:AFJSONParameterEncoding];
 
 NSMutableURLRequest *request = 
 [httpClient requestWithMethod:@"DELETE"
 path:DBBaseURLString
 parameters:nil];
 
 AFJSONRequestOperation *operation =
 [AFJSONRequestOperation JSONRequestOperationWithRequest:request
 success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {
 
 [self.delegate bugDeleted:YES];
                                         
 }
 
 failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON) {
 NSLog(@"%@",error);
                                           
 [self.delegate bugDeleted:NO];
                                               
 }];
 
 [operation start];
   
 }

Learn More

If you’re building a native iOS app, think about whether using a backend services platform with REST might work well for you. DreamFactory’s REST API made it very easy for Circa5 to develop Heisenbug without writing any server-side code or having to create their own API.

We’ve just covered a small part of what’s available in the DreamFactory backend. Other services are available for NoSQL data, binary documents, user permissions, and external services (like remote databases, file storage, and web services). For more information or to try it out for free, check out the following links. Thanks for reading and good luck with your apps!

DreamFactory Now Supports MongoHQ

              <div class="hs-migrated-cms-post"> 
               <p><img id="img-1378227826564" class="alignLeft" src="//staticassets.dreamfactory.com/images/team-ben.jpg?t=1527092313664&amp;width=86&amp;height=57&amp;name=team-ben.jpg" alt="BenBusse" width="86" height="57" border="0" srcset="//staticassets.dreamfactory.com/images/team-ben.jpg?t=1527092313664&amp;width=43&amp;height=29&amp;name=team-ben.jpg 43w, //staticassets.dreamfactory.com/images/team-ben.jpg?t=1527092313664&amp;width=86&amp;height=57&amp;name=team-ben.jpg 86w, //staticassets.dreamfactory.com/images/team-ben.jpg?t=1527092313664&amp;width=129&amp;height=86&amp;name=team-ben.jpg 129w, //staticassets.dreamfactory.com/images/team-ben.jpg?t=1527092313664&amp;width=172&amp;height=114&amp;name=team-ben.jpg 172w, //staticassets.dreamfactory.com/images/team-ben.jpg?t=1527092313664&amp;width=215&amp;height=143&amp;name=team-ben.jpg 215w, //staticassets.dreamfactory.com/images/team-ben.jpg?t=1527092313664&amp;width=258&amp;height=171&amp;name=team-ben.jpg 258w" sizes="(max-width: 86px) 100vw, 86px">We’re excited to announce that we’ve partnered with MongoHQ. Now you can connect directly to any MongoHQ database instance from the DreamFactory Services Platform (DSP). In a nutshell, this new feature makes accessing your MongoHQ data really easy.</p> 
               <ol> 
                <li>Securely store your MongoHQ credentials on a server. You never want to do this on the client for security reasons!</li> 
                <li>Make database calls to your MongoHQ database with a ready-to-go REST API. Now you don’t have to write your own API to access MongoHQ. DreamFactory creates the API for you. Just make an API call and get a JSON document back from the server.</li> 
                <li>Apply CRUD permissions at the collection level based on user role. This is a great feature if you need to apply different data permissions for different users in your application.</li> 
               </ol> 
               <p>In this blog post, we’ll show you how to connect to MongoHQ in 60 seconds, quickly browse the REST API to access your data stored in MongoHQ, and apply CRUD permissions for different user roles.</p> 
               <p>Before getting started, you’ll need to <a href="http://tour.mongohq.com/c/dreamfactory">sign up for a MongoHQ account</a> if you don’t already have one. The link above provides developers a $50 credit for opening a new account with MongoHQ. </p> 
               <p>Once you’ve logged into your DreamFactory Services Platform (DSP), you’ll need to configure your connection to MongoHQ. Make sure to <a href="https://www.dreamfactory.com/user/register">sign up for a DreamFactory account</a> if you don’t already have one. </p> 
               <h2>Set Up a MongoHQ Service</h2> 
               <p>Once you’re logged into your DSP as an administrator, start by clicking on the ‘Services’ tab on the left-hand side of the screen.</p> 
               <p><img id="img-1380909709522" src="//staticassets.dreamfactory.com/images/services_nav.png?t=1527092313664" alt="services nav" border="0"></p> 
               <p>Then click to Create New Service and fill out the required information about your MongoHQ database. See the screenshot below for an example.</p> 
               <p>Make sure to select ‘NoSQL DB’ in the Type list box and ‘MongoHQ’ in the NoSQL Type list box. You’ll then need to enter your MongoHQ connection string, database name, database username, and database password. Don’t forget to click the Save button at the bottom of the screen.</p> 
               <p><img id="img-1380931620416" src="//staticassets.dreamfactory.com/images/mongohq_setup1-resized-600.png?t=1527092313664" alt="describe the image" border="0"></p> 
               <h2>Browse the REST API for MongoHQ</h2> 
               <p>Once you’ve saved your MongoHQ connection settings, DreamFactory automatically creates a REST API for your MongoHQ database. To browse the API, just click on the ‘API Documentation’ tab on the left-hand side of the screen.</p> 
               <p><img id="img-1380931614416" src="//staticassets.dreamfactory.com/images/api_nav.png?t=1527092313664" alt="api nav" border="0"></p> 
               <p>Then find the API name that you used for MongoHQ (in our example we called the API ‘mongohq’ in the setup screen). You’ll see a total of 14 API calls that allow you to manipulate your MongoHQ data with standard GET, POST, PATCH, PUT, and DELETE verbs.</p> 
               <p><img src="//staticassets.dreamfactory.com/images/mongohq_api-resized-600.png?t=1527092313664" alt="mongohq api resized 600" border="0"></p> 
               <p>We use a great tool called Swagger for our API docs, which makes it really easy to interact with the API and inspect your MongoHQ data right from your web browser. We suggest taking a look at the live Swagger API before writing client-side code to access MongoHQ data from your app.</p> 
               <p>For example, we have a collection called 'zips' that contains demographic data by zipcode. Let’s say we want to run a simple filter on this data to look at cities in California with a population greater than 50,000. In this example, we’ll also limit the number of records returned to 5.</p> 
               <p>Doing so is easy. In the API docs, just navigate to the GET request for /mongohq/{table_name}. Then simply enter the name of the ‘zips’ collection, a simple filter string, and set the limit to 5.  </p> 
               <p><img src="//staticassets.dreamfactory.com/images/mongohq_params-resized-600.png?t=1527092313664" alt="mongohq params resized 600" border="0"></p> 
               <p>Now click the ‘Try it out!’ button and you’ll see the API request and response.</p> 
               <p><img src="//staticassets.dreamfactory.com/images/mongohq_json-resized-600.png?t=1527092313664" alt="mongohq json resized 600" border="0"></p> 
               <p>You can also make API calls from the command line. For example, you can use cURL to make a GET request on a particular MongoHQ table and append a BSON filter string in the request. You’ll see the corresponding JSON returned in the response.</p> 
               <p>And you can see the available MongoHQ API operations right in your browser using the DreamFactory Javascript SDK (for example with FireBug). </p> 
               <p><img id="img-1380932434548" src="//staticassets.dreamfactory.com/images/mongohq_firebug-resized-600.png?t=1527092313664" alt="mongohq firebug resized 600" border="0"></p> 
               <h2>Set Up CRUD Permissions to MongoHQ by User Role</h2> 
               <p>Lastly, if your application requires different data permissions for different user roles, you can easily configure CRUD access for your various MongoHQ collections.</p> 
               <p>For example, let’s say we have a ‘sales rep’ role and we only want to grant READ and WRITE access to a collection called ‘opportunities.’ We don’t want to allow a ‘sales rep’ user to be able to access other collections and we don’t want a ‘sales rep’ user to create or delete data in the ‘opportunities’ collection.</p> 
               <p>Setting up data access permissions only takes a few seconds. In this example, you’d simply create a Role called ‘sales rep.’ Then you’d click on the Roles tab on the left-hand side of the screen and assign ‘Read and Write’ service access to opportunities for the MongoHQ service that we just set up. That’s it!</p> 
               <p><img src="//staticassets.dreamfactory.com/images/mongohq_roles.png?t=1527092313664" alt="mongohq roles" border="0"></p> 
               <p><img src="//staticassets.dreamfactory.com/images/mongo_hq_crud1-resized-600.png?t=1527092313664" alt="describe the image" border="0"></p> 
               <h2>Conclusion</h2> 
               <p>We hope this quick tutorial shows how easy it is to set up MongoHQ in your DSP and access your data with REST and highly configurable CRUD permissions.</p> 
               <p>Please let us know what you think in the comments section!</p> 
              </div>

NoSQL, No Problem! – MongoDB Specifics

LeeHicksFor the third installment on this blog series on DreamFactory Services Platform (DSP) support for NoSQL, I wanted to include a little more information on how to use MongoDB as a service. As you may or may not know, MongoDB is the leading NoSQL database currently, and while it does come with a REST interface, it is simple and has no support for CRUD operations on the data itself and is generally used for monitoring or administrative tasks. We have taken one of the native drivers available and added a fully REST-based and “blended” (see earlier post) API for collection and document (otherwise known as table and record) operations.

MongoDB as a Service

As I mentioned in the earlier blog, DSPs allow you to “on-board” services that you want to use from your app, and at the same time give you user role-based access control of those services. These services can be local to the server running the DSP itself, or remotely running on a different server altogether. In the case of MongoDB, it can be loaded locally on the same server, remotely on a different server, or hosted on the web by a third party like MongoHQ or MongoLab.

To add your MongoDB as a service on your DSP, select the ‘Services’ menu option on the admin console and select ‘Create New Service’ on the far left if an empty form is not already present. Set the ‘Type’ drop down menu to ‘NoSQL DB’, select your desired service and API names. Select MongoDB for the ‘NoSQL Type’ drop down menu. You should see something like the following…

config

The Connection String can be a fully qualified connection containing the authentication (if required), host name, port (if not default) and database. If that is the case, then nothing is required for the ‘Database’, ‘Username’, and ‘Password’ fields.

Note the following exceptions…

  • If the username or password are required and contain special characters, particularly @, :, or / (see MongoDB documentation) in them, then they must not be in the connection string and must be added in the given form fields below.
  • If you desire for the database name to not be in the connection string, then it is required in the form field below. The service is defined for one database at a time.
  • If you don’t add ‘mongodb://’ to the beginning of the connection string it will be automatically added for you during use.
  • The connection string can be blank if the host name and port are not required because you are using mongo with the local default setup, i.e. localhost:27017. In this case, use the other fields for any required configuration, i.e. database.

Once you save the configuration, you can click the document icon to the right of the service listing and get the Swagger UI interface for documentation and testing or issue commands from our javascript sdk, your hosted or native app, or cURL, etc.

See my notes in the earlier blogs in this series here and here about how the HTTP verbs function and basic operations.

Table (Collection) Administration

To get a list of currently available collections and their details, just send a GET request to the service’s root. Add the URL parameter ‘include_properties’ to see MongoDB-specific details about each collection, including any default or provisioned indexes.

curl 'https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo?include_properties=true'

The response will look like the following…

{
    "table": [
        {
            "name": "test",
            "indexes": [
                {
                    "v": 1,
                    "key": { "_id": 1  },
                    "ns": "test.test",
                    "name": "_id_"
                }
            ]
        },
        {
            "name": "zipcodes",
            "indexes": []
        }
    ]
}

Use this same data format for creating new tables, or deleting existing tables. To create a new table, use the following…

curl -X POST https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo \
     -H 'Content-Type: application/json' \
     -d '{ "table": [ { "name": "test" } ] }'

Note: Currently, you can’t create or update indexes or cap the size of new collections through this interface. This functionality is planned for release at a later time.

You can also use the ‘names’ url parameter in the retrieve request or for a delete request to delete existing tables.

curl -X DELETE 'https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo?names=test'

All table operations return, at a minimum, an array of table names affected by the operation. For example, the above two operations return…

{
  "table": [
      {
	    "name": "test"
	  }
    ]
}

Record (Document) Administration

In the previous blog, I described the “blended” REST API operations for record CRUD operations. As mentioned, all of those apply to using MongoDB as a service. However, for those of you who enjoy MongoDB’s json format, particularly for query request, DreamFactory also supports most of MongoDB’s json query document format.

Query Operations

Here are the available options for a GET request on a particular table in the DreamFactory REST API for NoSQL DBs.

get

The table name is sent as part of the url, while the other options are typically sent as url parameters. If you would like to use MongoDB’s json query document format, you can send the ‘filter’ parameter in json format as part of the posted data. For example, if we want to find the first 3 records in the zipcodes table that have a population of over 20,000 people, returning only the city, state and percentage over 40 years of age. If you are familiar with SQL, this would look like “SELECT city,state,PopulationOver40 from zipcodes WHERE pop > 20000;”. Here is what it looks like in our REST API using the “blended” approach with a simple SQL filter.

curl 'https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes?filter=pop+%3E+20000&limit=3&fields=city%2Cstate%2CPopulationOver40'

Using MongoDB’s native json query document format, we will post the ‘filter’ parameter as json data. The ‘limit’ and ‘fields’ parameters will continue to be passed as URL parameters. With cURL, we can accomplish this by using the -X GET option to pass the data.

curl -X GET 'https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes?limit=3&fields=city%2Cstate%2CPopulationOver40' \
     -H 'Content-Type: application/json' \
     -d '{"filter": { "pop": { "$gt": 20000 } } }'

Other utilities or clients may require the HTTP method to be specified as POST and include either a URL parameter ‘method=GET’ or an extra header value of ‘X-HTTP-METHOD = GET’ (shown here for clarity).

curl -X POST 'https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes?limit=3&fields=city%2Cstate%2CPopulationOver40' \
     -H 'X-HTTP-METHOD: GET' \
     -H 'Content-Type: application/json' \
     -d '{"filter": { "pop": { "$gt": 20000 } } }'

In all three cases, this is what gets returned…

{
  "record": [
    {
      "_id": "28659",
      "city": "NORTH WILKESBORO",
      "state": "NC",
      "PopulationOver40": 9.19
    },
    {
      "_id": "31201",
      "city": "HUBER",
      "state": "GA",
      "PopulationOver40": 99.877
    },
    {
      "_id": "71291",
      "city": "WEST MONROE",
      "state": "LA",
      "PopulationOver40": 80.099
    }
  ]
}

Passing the filter this way opens up use of the query operators supported natively in MongoDB. Besides record retrieval, these query documents can also be used as filters when updating or deleting records, along with the other methods mentioned in the ‘Updating and Merging Into Records’ section of the previous post.

Updating Records

As mentioned in the previous blog, updates to NoSQL records using the API generically take on one of two forms; either replacing the whole record by posting the new name-value pairs along with any old name-value pairs that you want to keep to that record’s identifier (using PUT requests), or by merging changes to exiting records by posting only the changed or new name-value pairs (using PATCH requests).

For the second scenario, merging changes, MongoDB also supports specialized update operators that facilitate changes to record data in various ways. These can also be passed in the posted data as part of the records themselves when using the json format.

For example, consider a record in the zipcodes table with the following data…

    {
      "_id": "29684",
      "city": "STAR",
      "state": "SC",
      "pop": 2889
    }

Lets say that we need to correct the spelling of the city name and increment the population by 5. We could do this by sending the changed name value pairs in a PATCH request to merge in the changes as follows…

curl -X PATCH 'https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes/29684' \
     -H 'Content-Type: application/json' \
     -d '{ "city": "STARR", "pop": 2894 } } }'

Obviously, for the population increase to be accurate we would have to pull a copy of the value to the client to get the latest before the modification, not ideal. Using the native MongoDB update operators, we can send it as follows…

curl -X PATCH 'https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes/29684' \
     -H 'Content-Type: application/json' \
     -d '{ "$set": { "city": "STARR" }, "$inc": { "pop": 5 } }'

Either way yields the results…

    {
      "_id": "29684",
      "city": "STARR",
      "state": "SC",
      "pop": 2894
    }

What Next?

Stay tuned for more specifics on other NoSQL vendors, like AWS DynamoDB, etc.

Why You Shouldn’t Build Your Own REST API

 
BenBusseWhat’s the story behind the DreamFactory Services Platform? We make applications ourselves on cloud platforms like Salesforce, Windows Azure, and AWS. Every new application we created for our customers required the same manual steps:
  • Set up backend databases, schema, and file storage
  • Create a user management system with secure authentication
  • Design and create backend services for data, files, and external APIs
  • Write our own REST API to access all these services
  • Integrate the frontend application with these backend services
  • Test all of that integration end-to-end
Ouch! All that time spent creating the backend services and API took away valuable time creating the actual application that customers would be using everyday. We searched for an open source solution that could solve this problem. Alas, it didn’t exist. So we decided to build it. We realized that other app developers faced the exact same problems and could benefit from our work. So it made perfect sense to open source it. Many of our customers are large enterprises with sophisticated requirements, especially around security. The platform had to satisfy several goals:
  1. Dramatically simplify life for frontend developers. The platform should eliminate the need to write any server-side code.
  2. Support HTML5 and native mobile applications running on performance and bandwidth-constrained phones and tablets.
  3. Provide a comprehensive palette of backend services and a unified REST API to power sophisticated, data-driven applications at scale.
  4. Provide world-class security that large enterprises could adopt.
  5. Provide open source flexibility. A developer or sys admin should be able to install the DreamFactory software package in the cloud or on premise.
The unified REST API mentioned in goal 3 above is a key feature of the platform. Now you don’t have to write your own REST API. It’s automatically created for every backend service that your application needs. The API includes 123 standard GET, POST, PUT, and DELETE calls for:
  • /user – 11 API calls for user authentication, registration, profiles, and sessions
  • /system – 45 API calls for managing apps, app groups, email, roles, services, and users
  • /app – 16 API calls for application containers, files, and folders
  • /db – 8 API calls for database CRUD operations
  • /doc – 16 API calls for document containers, files, and folders
  • /email – 1 API call to send email
  • /lib – 16 API calls for lib containers, files, and folders
  • /schema – 10 API calls for managing schema
And every time you add a new service, the corresponding REST API for that service is automatically created and documented. For example, say you connect to a MongoDB database with DreamFactory. updateservice The new API ‘/mongo’ is automatically created and documented. Presto, now you have a REST API to access your remote MongoDB database from the client! mongodb Before you start building anything, spend 5 minutes browsing the API. The API is documented with an awesome tool called Swagger. Swagger lets you try out live API calls right in your browser.  It’s interactive, so you can quickly learn the capabilities of the API without writing a line of application code. Try it out! You can browse the API in two places: on our website and in the API Documentation tab of the admin console. apidocumentationtab Also check out Jason’s recent blog post on getting started with the DreamFactory API. Have fun with the API and let us know what you think!

NoSQL, No Problem! – Operation Specifics

 
LeeHicksAs a continuation to my initial blog on NoSQL support on the DreamFactory Services Platform (DSP), I would like to give you a little more information on how to use the NoSQL service operations. Designed to be flexible and powerful, yet still adhere to simple REST principles, there are several options for CRUD operations available in the DSP REST API for NoSQL services. Once you have configured your NoSQL service, you can click the document icon to the right of the service listing on the admin console and get the Swagger UI interface for documentation and testing as seen below. DreamFactory MongoDB Rest API Notes about this blog:
  1. I am using MongoDB as our example service but all supported NoSQL types behave similarly, with some exceptions.
  2. Also, I am using JSON format everywhere here, but XML is also supported.
  3. In the cURL request, not all headers are included in request, i.e. authentication, app name, etc. You can use the Swagger UI to perform many of these same calls much easier.
  4. See my notes in the earlier blog about how the HTTP verbs function.

Table Administration

As you see from the list above, all table administration operations utilize the root of the service for the URL. All tables are identified by a ‘name’ field in the operations. All posted or returned data is transmitted as a single object representing a table containing a ‘name’ field at a minimum, or an array of table objects returned as a ‘table’ field value, except where noted. Retrieving Tables Every DSP service, when queried at its root with no extra parameters, returns an array of ‘resources’ available by that service. In the SQL and NoSQL case, these resources are table names. To get a list of currently available tables, just send a GET request to the service’s root, which, if you are using one of our hosted DSPs, looks like this.
curl https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo
The response looks like…
{
  "resource": [
    {
      "name": "test"
    },
    {
      "name": "zipcodes"
    }
  ]
}
To add vendor-specific details to the output, just add the query parameter ‘include_properties’ set to true. In this case, we get MongoDB-specific details about each collection.
curl https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo?include_properties=true
The response looks like…
{
    "table": [
        {
            "name": "test",
            "indexes": [
                {
                    "v": 1,
                    "key": {
                        "_id": 1
                    },
                    "ns": "test.test",
                    "name": "_id_"
                }
            ]
        },
        {
            "name": "zipcodes",
            "indexes": []
        }
    ]
}
Additionally, there is a ‘names’ query parameter accepting a comma-delimited list of table names that allows the client to selectively retrieve information about specific tables
curl https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo?include_properties=true&names=test,zipcodes
Creating Tables Use the POST operation to create one table or multiple tables at once. At a minimum, a value for the ‘name’ field is required to create a table. Additional vendor-specific fields may also be sent. Some vendors may require additional fields. Again, a single object or an array of objects is permitted in the posted data. The output will reflect the input, i.e. an array posted will result in an array received. To create a new table, use the following…
curl -X POST https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo \
     -H 'Content-Type: application/json' \
     -d '{ "table": [ { "name": "test", … } ] }'
The response looks like…
{
    "table": [
        {
            "name": "test",
            "indexes": [
                {
                    "v": 1,
                    "key": {
                        "_id": 1
                    },
                    "ns": "test.test",
                    "name": "_id_"
                }
            ]
        }
    ]
}
Updating Tables Most of the NoSQL vendors accept little to no updates on the meta-data part of the tables. Some do however, and those would be accessed as follows…
curl -X PATCH https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo \
     -H 'Content-Type: application/json' \
     -d '{ "table": [ { "name": "test", “indexes”: {…} } ] }'
Deleting Tables Obviously this should be approached with caution. You can use the ‘names’ query parameter in a delete request to delete existing tables.
curl -X DELETE https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo?names=test
An additional option for delete would be to post the same data format as the other operations.
curl -X DELETE https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo \
     -H 'Content-Type: application/json' \
     -d '{ "table": [ { "name": "test" } ] }'

Record Administration

The DSP REST API provides the following ways for your app to retrieve, create, update and delete data from the tables. The table name is sent as part of the URL, while the other options are sent as query parameters or as part of the posted data. Many of the query parameters can also be sent via the posted data. One special query parameter available on all requests is the ‘fields‘ parameter. This dictates which fields will be returned for the affected records of the operations. It accepts a comma-delimited string of field names when passed as a query parameter, or an array of field names when passed in posted data. For GET request, this parameter defaults to returning all fields, i.e. a ‘*’ value, while all other request types return only the record identifying fields by default. This saves the client from having to do an additional round-trip call to get things like updated or auto-filled field values, or to get a list of records changed when updating by a filter. Again, where applicable, a single object or an array of objects is permitted in the posted data. The output will reflect the input, i.e. an array posted will result in an array received. Creating Records To create a single record …
curl -X POST https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes \
     -d '{ "_id": "95008", "state": "CA", "pop": 39968 }'
To create multiple records…
curl -X POST https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes \
     -d '{ “record” [ { "_id": "95008", "state": "CA" }, { "_id": "30022", "state": "GA"} ] }'
If the DB vendor does not automatically create the identifying fields (primary key), it must be included in the POST request. The applicable identifying fields are always returned for successfully created records on a create request. Retrieving Records There are many ways in which an app can retrieve data through our API. Depending on your app you may want to use one or all of these. Note that the first three also make use of the optional query parameter ‘id_field’ which allows the client to state which field is used as the identifying field for the records in that table. Here is a quick look at the retrieval options.
  • By a single record identifier – In this case, the identifying field (think primary key) value is passed as part of the URL after the table name. This will return a single record or a 404 – Not Found error.
curl https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes/95008
Returns this…
{
    "_id": "95008",
    "city": "CAMPBELL",
    "state": "CA",
    "pop": 39968
}
  • By a list of record identifiers – This case uses the ‘ids’ query parameter sent as a comma-delimited string of id values, or ‘ids’ field sent as comma-delimited string or an array of id values in posted data. If a large number or ids, or the id values are long or have special characters in them, it would be better to pass them as posted data.
curl https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes/?ids=95008,30022
Returns this…
{
    "record": [
        {
            "_id": "95008",
            …
        },
        {
            "_id": "30022",
            …
        }
    ]
}
  • By record – This case allows you to post a single record or an array of records that at a minimum include the identifying field(s) and values. This is useful in retrieving records with more than one key, or for updating a stash of records that you have old or partial data for already. (Note: The below cURL call works for *nix and mac, for Windows, use -X POST -H “X-HTTP-Method: GET” instead of -X GET.)
curl -X GET https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes \
     -d '{ “record” [ { "_id": "95008", … }, { "_id": "30022", … } ] }'
  • By filter – This is the most powerful option for data retrieval. The NoSQL service allows the client to use a simple SQL-like filter string passed as a query parameter (url-encoded), or in some vendor cases, native filters in various formats (like MongoDB’s json format, more on this in later vendor-specific blogs).
For our simple SQL-like filter string, comparison operators supported for every vendor are =, !=, >, >=, <, <=.; Or given as their SQL short form ‘ eq ‘, ‘ ne ‘ (or ‘ <> ‘), ‘ gt ‘, ‘ ge ‘, ‘ lt ‘, ‘ le ‘. Depending on the db vendor, others such as contains, like, and begins-with may also be supported. Spaces surrounding the operator is required. String values on the right side of the comparison must be within single or double quotes. Depending on the db vendor, logical comparisons (AND, OR, NOT) are also supported in order to build out more complicated queries. Other ‘filter helper’ query parameters are also available.
  • limit – Defaults to return all, accepts an integer greater than 0 to limit the number of returned records.
  • order – Accepts a field name followed by space and then ASC or DESC to order the returned records.
  • offset – Accepts an integer greater than 0 to skip that many records in the response. This is useful for setting up paging through records, but may not be supported by all vendors.
For example, if we want to find the first 3 records in the zipcodes table information that have a population of over 20,000 people, returning only the city and state. If you are familiar with SQL, this would look like “SELECT _id,city,state from zipcodes WHERE pop > 20000;”. Here is what it looks like in cURL.
curl https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes?filter=pop+%3E+20000&limit=3&fields=city%2Cstate
And this is what gets returned…
{
    "record": [
        {
            "_id": "28659",
            "city": "NORTH WILKESBORO",
            "state": "NC"
        },
        {
            "_id": "31201",
            "city": "HUBER",
            "state": "GA"
        },
        {
            "_id": "71291",
            "city": "WEST MONROE",
            "state": "LA"
        }
    ]
}
Updating and Merging Into Records The same options available for retrieving data also apply to updating records. As mentioned in the earlier blog, the PUT HTTP verb is used when the whole record is to be replaced with the posted data. The PATCH HTTP verb is used when the client only wants to send the changing fields to the server. The same array of records or a single record format with changes is supported for updating records. To replace the whole record at id of 95008, send the request like…
curl -X PUT https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes \
     -d '{ "_id": "95008", "city": "Campbell", "state": "CA", "pop": 40102 }'
If you only want to merge changes into that record without having to resend everything else, then send the request like…
curl -X PATCH https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes \
     -d '{ "_id": "95008", "pop": 40102 }'
If you only want to update or merge data for one record, and it can be identified by a single key field, then you could also add the id to the end of the URL, and pass only the fields that require change.
curl -X PATCH https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes/29684
     -d '{ "pop": 40102 }'
Two other methods of merging data are by id list or filter (using “ids” or “filter” url parameters mentioned above for retrieving records). In these cases, if not natively supported, the server will query the table for the filtering results, merge in the record changes and push the changes back in an update request. This is an easy way of updating multiple records with the same field-value changes. Deleting Records Deleting supports the same options as updating. The only difference is, when using the id, ids or filter options, no posted data is required. Obvious, right? Using the ‘fields’ query parameter, the client can request the full or partial records before they are deleted from the system. What Next? Stay tuned for more specifics on each NoSQL vendor, next up more on MongoDB.

NoSQL, No Problem!

 
LeeHicksYou may have already seen that DreamFactory Service Platform (DSP) supports SQL databases with its REST API, but did you know that we also support most of the popular NoSQL databases? Currently, DSPs support Amazon Web Services’ SimpleDB and DynamoDB, Windows Azure’s Tables, MongoDB, MongoHQ, and CouchDB.

While the technology behind, and the interface in front of, each of these services vary, some dramatically, they each support storing data and lots of it. Some already have a REST or somewhat  REST-like HTTP interface to access them, which may mean storing your access keys or authorization information on the client and passing them along with every call. Some do not have an HTTP interface at all and must be accessed through server-side code that you have to go find and install or write yourself. Some allow you to design the primary keys or indexes of your storage tables, some do not. Some support JSON natively, some do not. In fact, the formats supported for data entry and retrieval (think filtering) vary almost as widely as the number of vendors. Get the drift? Different creators, different creations. But you say “I just want a simple way for my app to get my data in and out, quickly and easily.” That’s where I think we can help.

NoSQL as a Service

DSP Services allow you to “on-board” web-based or locally running services that you want to use from your app and access from one authenticated REST API. The access to these services can be controlled by app and by user role. There is more on this and how to provision these services in our documentation here.

The same goes for the supported NoSQL services. We securely store any credentials required to access your storage on the server-side, relieving your app from maintaining it. We remove any cross-site hoops your app would have to handle by making it a “native” REST service on the DSP. We even go a step further and provide a common, fully REST-based API to each of the supported NoSQL service types and use a common data format layout, using JSON, for data entry and retrieval. Where we were able, we also provide a common SQL-like filtering language, making it dirt simple to find the data you need.

We like to call this our “blend layer”. The idea is, just like a DAO (data access object), we simplify the process to use your data. In fact, we have made this so “blended” it is nearly identical to our SQL DB REST API, which we think is powerful but simple to use. So how does this help you? Well, you write the app once, and store your data in your favorite storage. If or when your “favorite” storage changes (i.e. if something cheaper or faster comes along), your app and data will only require minor changes (if any) to keep chugging along.

So what does this wonderful interface look like? Well, since you asked so nice and all. Once you get your service setup on your DSP (using the documentation mentioned earlier, or the vendor-specific follow-up blogs to this blog), you can use our Live API documentation interface to see how it works. Of course, you can use cURL or your favorite REST testing browser app, it is up to you. Since I am such the promotionalist, I’ll use the Swagger interface in this blog to show you.

As with the other DSP services, we put a mostly standard meaning to the HTTP verbs in our REST API. Here is the how they are used for NoSQL services (using “mongo” as our demo service).

DreamFactory MongoDB Rest API

Note that the ‘POST’ is used for creating new entities, ‘PUT’ is used for replacing the whole content of an entity and ‘PATCH’ is used to merge in changes to an entity (‘PATCH’ and ‘MERGE’ HTTP verbs are used interchangeably in the REST API). Also, as with the other DSP services, if your transport layer only accepts ‘GET’ and ‘POST’ verbs, using a ‘POST’ command while setting either a url parameter, ‘method=PATCH’, or a header, X-HTTP-Method = PATCH will “tunnel” the command through properly.

Table Administration

Azure calls them “tables”. So does AWS, that is for DynamoDb, but for SimpleDb, they are called “domains”. For CouchDb, they are called “databases”, not to be confused with MongoDb’s “databases” which house what they call “collections”. See what I mean?? While each vendor calls them something different, we will refer to them collectively as “tables”. These are the things that group your sets of data together or partition them, however you see it. You can actually manage most aspects of the table administration through the REST API.

To get a list of currently available tables, just send a GET request to the service’s root, which, if you are using one of our free-trial DSPs, looks like this.

https://dsp-mydspname.cloud.dreamfactory.com/rest/my_service_api_name_here

Every DSP service, when queried at its root with no extra url parameters, returns an array of resources available by that service. In the SQL and NoSQL case, these resources are table names.

{
   "resource": [
      {
         "name": "test"
      },
      {
         "name": "zipcodes"
      }
   ]
}

To include vendor-specific table information, if any is available, you can send ‘include_properties=true’ as a url parameter in the request. You can also limit the returned data by specifying the tables by name, using the ‘names=table1,table2,table3’ url parameter. The returned data will include a table array, where each object in the array includes at least the ‘name’ field and value, as well as, any available vendor-specific properties of the table, i.e. keys, etc.

https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo?include_properties=true&names=test%2Cuploads
{
   "table": [
       {
           "name": "test",
           "indexes": [
               {
                   "v": 1,
                    "key": {
                       "_id": 1
                   },
                   "ns": "test.test",
                   "name": "_id_"
               }
           ]
       },
       {
           "name": "uploads",
           "indexes": []
       }
   ]
}

Use this same data format for creating new tables, or updating and deleting existing tables. You can also use the ‘names’ url parameter in a delete request to delete existing tables.

Record Administration

Again, whether they are called “entities”, “documents”, or “items”, we collectively call them “records”. These are the groups of name-value pairs that make up your app’s data. The DSP REST API provides the following ways for your app to retrieve, create, update and delete data from the tables.

Retrieving Records

When it comes to making your NoSQL data available to your app, our REST API provides several different methods to get just the data sets you want, when you want it. Here are the available options for a GET request on a particular table.

table get request

The table name is sent as part of the url, while the other options are sent as url parameters. If url parameters are not your cup of tea, you can send most of them as posted data (using POST request with X-HTTP-METHOD = GET as mentioned earlier. Note that the “record” parameter can only be sent in this way.). Most of them are self explanatory, but a few may need clarification.

  • ids” – This is a comma delimited list of unique identifiers (think primary key) values for retrieving multiple records at a time. If passed as POST data, this can also be an array of ids. In the case where they is no static identifier field (DynamoDb), use  of the “id_field” parameter is necessary to indicate which field these values belong to.
  • filter” – This is where you use our simple SQL-like filter string. Comparison operators supported on every platform are ‘ = ‘, ‘ != ‘, ‘ > ‘, ‘ >= ‘, ‘ < ‘, ‘ <= ‘; or as their SQL short form ‘ eq ‘, ‘ ne ‘, ‘ gt ‘, ‘ ge ‘, ‘ lt ‘, ‘ le ‘. Depending on the db vendor, others such as ‘ contains ‘, ‘ like ‘, and ‘ begins_with ‘. Spaces surrounding the operator is required. String values on the right side of the comparison must be within single of double quotes. Depending on the db vendor, logical comparisons (AND, OR, NOT) are also supported. More on the differences in later blogs. The whole filter string must be url-encoded.
  • record” – Used only in POST data, this option allows you to send partial (minimally the identifier fields) or complete records back to the database to be “refreshed” with the latest values. This is also helpful in instances where the table has multiple indexes, i.e. DynamoDb and Azure Tables.

For example, if we want to find the first 3 records in the zipcodes table information that have a population of over 20,000 people, returning only the city, state and percentage over 40 years of age. If you are familiar with SQL, this would look like “SELECT city,state,PopulationOver40 from zipcodes WHERE pop > 20000;”. Here is what it looks like in our REST API.

https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes?filter=pop+%3E+20000&limit=3&fields=city%2Cstate%2CPopulationOver40

And this is what gets returned…

{
 "record": [
   {
     "_id": "28659",
     "city": "NORTH WILKESBORO",
     "state": "NC",
     "PopulationOver40": 9.19
   },
   {
     "_id": "31201",
     "city": "HUBER",
     "state": "GA",
     "PopulationOver40": 99.877
   },
   {
     "_id": "71291",
     "city": "WEST MONROE",
     "state": "LA",
     "PopulationOver40": 80.099
   }
 ]
}

Creating Records

Use this same data format above for creating new records. One could post an array of records, or a single record like this…

POST https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes
   {
     "_id": "29684",
     "city": "STARR",
     "state": "SC",
     "pop": 2889,
     "PopulationOver40": 82.4
   }

If the DB vendor does not automatically create the primary key, it must be included in the POST request. The applicable keys are always returned for successfully created records on a create request.

Updating and Merging Into Records

The same array of records or a single record format with changes is supported for updating records. Using the PUT HTTP verb as mentioned above will replace the whole record with the posted data if found by matching identifiers in the record.

PUT https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes
   {
     "_id": "29684",
     "city": "STARR",
     "state": "SC",
     "pop": 2890,
     "PopulationOver40": 79.4
   }

If you only want to merge changes into a record without having to reset everything, then use the MERGE or PATCH HTTP verb and send only the changes along with the identification fields.

PATCH https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes
   {
     "_id": "29684",
     "pop": 2890,
     "PopulationOver40": 79.4
   }

If you only want to update or merge data for one record, and it can be identified by a single key field, then you could also add the id to the end of the url, and pass only the fields that require change.

PATCH https://dsp-mydspname.cloud.dreamfactory.com/rest/mongo/zipcodes/29684
   {
     "pop": 2890,
     "PopulationOver40": 79.4
   }

Two other methods of merging data are by id list or filter (using “ids” or “filter” url parameters mentioned above for retrieving records). In these cases, if not natively supported, the server will query the table for the filtering results, merge in the record changes and push the changes back in an update request. This is an easy way of updating multiple records with the same field-value changes.

Deleting Records 

Deleting is similar to updating records but no posted data is required.

More to Come…

Stay tuned for a series of blogs on each of the supported NoSQL services and how to use them in your next awesome app.

Getting Started with the DreamFactory API

 

JasonSykesIn this blog post, we’ll show you how to make some simple API calls with the DreamFactory API. The API requires SSL3. If you plan on making requests from cURL, or your native SDK, make sure you are using version 3.

For example, in cURL:

curl -3 url

Your “api key” to talk to our API is your App Name as defined in the DreamFactory Administration Console. For each request, you can:

  1. Append app_name=yourappname to the query string or
  2. Send a request header called X-DreamFactory-Application-Name with the value of your app name

Working with User Sessions (/user/session)

Authentication 

If your application is not part of the guest user’s role, then access to any service or data components will require authentication. To authenticate a user, simply POST a JSON string to /user/session that takes on the following format:
{“email”:”email_value”, “password”:”password_value”}
For example:
curl -k -3 -X POST https://dsp-yourdsp.cloud.dreamfactory.com/rest/user/session -H "X-DreamFactory-Application-Name: todojquery" -d "{ \"email\" : \"[email protected]\", \"password\" : \"foo\" }"
This will return one of four responses:  
  • A 201 as the session has been created. In the response, you’ll see a session_id has been created. Very Important: For all future requests to the API, you’ll need to pass this value as a new request header called X-DreamFactory-Session-Token
 
  • A 401 if the login information did not match any records on the backend
 
  • A 403 if CORS has not been enabled and you’re trying to use the API cross-domain
 
  • A 400 if you did not include your API Key (App Name)           

You can append app_name=yourappname to the query string or send a request header calledX-DreamFactory-Application-Name.

To refresh a user’s session, or just check to see if they have one, simply do a GET on /user/session.

Using our local DB service (/db/[table name]/[id])

As you’d expect, getting to your data is just a URL away. 

Retrieving Data 

To access any table available to your role, use the following format. In this example, we make a request for the todo table:

GET https://dsp-yourdsp.cloud.dreamfactory.com/rest/db/todo
For example:
curl -3 https://dsp-yourdsp.cloud.dreamfactory.com/rest/db/todo?app_name=todoangular

Important Note: Don’t forget your X-DreamFactory-Application-Name, and if a session is required, your X-DreamFactory-Session-Token header.

If you are doing this cross-domain, enable CORS in the System Configuration section of the DreamFactory Administration Console.

Filtering Data

To filter data by field, add an additional url param (filter) as follows. In this example, we make a request for the todo table:

https://dsp-yourdsp.cloud.dreamfactory.com/rest/db/todo?app_name=todoangular&filter=complete=true

This API call returns all todo items that are complete.

Attaching Schema

To attach your table’s schema as metadata in the response, just add a url param, like this:

GET https://dsp-yourdsp.cloud.dreamfactory.com/rest/db/todo?include_schema=true

Important Note: Don’t forget your X-DreamFactory-Application-Name, and if a session is required, your X-DreamFactory-Session-Token header.

Enjoy!  If you have any issues, head over to http://dreamfactorysoftware.github.io

Adding a Remote Web Service to Your DSP

 
ToddAppletonThe DreamFactory Services Platform (DSP) provides a flexible and powerful backend for your mobile apps. If you don’t have a DreamFactory account, sign up here. One of the key features is the ability to integrate any number of remote web services, and access them directly through the DreamFactory REST API. In this blog post we’ll walk through the process of setting up and invoking a remote web service. Continue reading “Adding a Remote Web Service to Your DSP”