Demo – How to Make Relationship Queries with DreamFactory’s REST API

 
BenBusseDevelopers often ask how to query related data with DreamFactory’s REST API. For example, how do you fetch a record and its related children as JSON in a single API call? In this post, we’ll demonstrate related data queries with a real example. Also check out these related blog posts. Why We Like Swagger for API Docs covers how to use our live API docs to explore the API. To learn more about how Modus Create built the Address Book application with DreamFactory, check out the DreamFactory & Modus Create Case Study. 1. To follow along, you’ll first need a DreamFactory account. You can sign up for an account here. 2. Log in to your account, go to your DSP admin console, and click on the Applications tab on the left hand side. Swagger blog 1 3. Click ‘Import New App’ in the upper right corner and choose the Address Book sample app. This contains related schema for both parent-child and related records via junction table relationships. Swagger blog 2 Once you’ve imported the application, you can preview it in your DSP. Swagger blog 3 Use your DSP admin user name and password to log in to the Address Book app and see the app. Swagger blog 4 4. Go to the ‘Manage Schema’ and ‘Manage Data’ tabs to peruse the table layouts and sample data if you wish. Swagger blog 5 The Contacts table stores basic information about each contact. ContactInfo stores contact information like phone, email, and address. A Contact can have many child ContactInfo records. The ContactGroups table stores information about Groups. A Contact can belong to any number of Groups. ContactRelationships is a junction table that stores the associations between Contacts and Groups. Swagger blog 6 5. Click ‘API Documentation’ to load Swagger REST API interface. Swagger blog 7 Select ‘schema’ -> GET /schema/{table_name} to expand. Swagger blog 8 In the ‘table_name’ field type ‘Contacts’ and click the ‘Try it out’ button. Swagger blog 9 In the Response Body, you’ll see the field section and the related objects. Note the relationship names in the JSON (highlighted in red below) and the key value pairs that describe each relationship. Swagger blog 12 6. Back in the main Swagger API Documentation, Select ‘db’ -> GET /db/{table_name}. Swagger blog 10 In the ‘table_name’ field type ‘Contacts’. In the ‘ids’ field enter a couple of ids from the sample data e.g. 1,3). In the ‘related’ field type ‘ContactInfos_by_contactId,ContactGroups_by_ContactRelationships’. This will pull both the child info records and the groups related by junction. Click ‘Try it out’ and note the URL call and results. Swagger blog 11 The Request URLlooks like this, where ‘foo’ is the name of your own DSP. http://dsp-foo.cloud.dreamfactory.com/rest/db/Contacts?ids=1%2C3&related=ContactInfos_by_contactId%2CContactGroups_by_ContactRelationships Take a look at the Response Body and notice the JSON structure for child data, in this example multiple ways to contact a person. Also notice how the JSON encodes the associations between a Contact and zero or more Contact Groups to which the Contact belongs. Swagger blog 13 We hope this overview has shown you how to make sophisticated queries with DreamFactory’s REST API! Please comment if you have any feedback or questions.

jQuery App Tutorial with the DreamFactory REST API

 
ToddAppletonThis blog post is about building a to do list app using the DreamFactory Services Platform (DSP) for the backend. We provide a comprehensive REST API and an easy-to-use JavaScript SDK for making API calls. This example will show you how to authenticate to your DSP and then make simple CRUD calls using the SDK. If you don’t have a DreamFactory account, sign up here. This app is available on new DSP’s as a sample app named “Todo List jQuery w/SDK”. If you don’t have it installed you can import it from GitHub using the apps tab in the admin console.

Code Walkthrough

This app consists of three files plus the SDK. index.html – loads jQuery and Twitter Bootstrap and has a table to hold the list. It has an input field and add button for adding items to the list. app.js – functions for creating, retrieving, updating, and deleting list items and building the list. style.css – simple styling The complete source code for this app is available on GitHub at https://github.com/dreamfactorysoftware/app-todo-jquery.  You can go there to clone this repo to your local machine. In the SDK /lib directory there is a file sdk_init.js. This is where you specify the name of your app and the DSP your app is talking to. For hosted apps that use the DSP for app file storage you could set dsp_url to use location.protocol and location.host and it would always init the SDK to use the “current” DSP. If not a hosted app then dsp_url should be set to the DSP you want to use for that app. When the SDK has been initialized the ‘apiReady’ event is triggered. At that point the app is ready to make API calls. The first thing we should do is check for a valid session and, if none exists, try to log in. The doLoginDialog() and login() methods take care of this.
$(window).on("apiReady", function () {

    checkSession();
});

function checkSession() {

    $("#loading").show();
    // check for existing session, relevant when code is hosted on the dsp
    window.df.apis.user.getSession({"body": {}}, function (response) {
        $("#loading").hide();
        // existing session found, assign session token
        // to be used for the session duration
        var session = new ApiKeyAuthorization(
            "X-Dreamfactory-Session-Token",
            response.session_id,
            'header');
        window.authorizations.add("X-DreamFactory-Session-Token", session);
        runApp();
    }, function (response) {
        $("#loading").hide();
        // no valid session, try to log in
        doLogInDialog();
    });
}
After a session has been established we call runApp() which is the main entry point into the app. The function getRecords() uses the SDK to retrieve all records from the database table named todo. You could easily use NoSQL storage such as MongoDB by adding it as a new service on your DSP, then changing df.apis.db to df.apis.mongodb in each SDK call.
function runApp() {

    // your app starts here
    getRecords();
}

function getRecords() {

    window.df.apis.db.getRecords({"table_name":"todo"},
    function (response) {
        buildItemList(response);
    }, crudError
    );
}
}
On success, an array of records is returned such as
{
   "record":[
      {
         "id":"35",
         "name":"item 1",
         "complete":false
      },
      {
         "id":"36",
         "name":"item 2",
         "complete":true
      },
      {
         "id":"38",
         "name":"item 3",
         "complete":false
      }
   ]
}
buildItemList() uses this array to populate a table inside the list-container div. Column 1 of the table is for the delete icon. Column two is for the item name. The id of each item is stored in data-id and used when the item is updated or deleted. Note the onclick handlers being set up for the update and delete ui elements. This is where the completion is toggled.
function buildItemList(json) {

    var html = '';
    if (json.record) {
        json.record.forEach(function (entry) {
            var name = entry.name;
            var id = entry.id;
            html += '';
            html += '';
            if (entry.complete === true) {
                html += '' + name + '';
            } else {
                html += '' + name + '';
            }
            html += '';
        });
    }
    $('table').html(html);
    $('#list-container .item').click(function (e) {
        var id = $(this).data('id');
        var complete = $(this).hasClass('strike');
        updateRecord(id, !complete);
    });
    $('#list-container i').click(function (e) {
        var id = $(this).data('id');
        deleteRecord(id);
    });
}
When you enter a new item and click the Add Item button, the function createRecord() is called. It grabs the item name from the input field and, if not empty, calls the SDK createRecords() method. The POST data is passed in as the “body” field and is an array of JS objects. In this case there’s only one element in the array. Since it’s a new item we will set the complete field to false. On success the input field is cleared and the list is rebuilt. To keep things as simple as possible we are calling getRecords() to retrieve and rebuild the entire list rather than updating the DOM with only the items that changed.
function createRecord() {

    var name = $('#itemname').val();
    if (name === '') return;
    var item = {"record":[
        {"name":name, "complete":false}
    ]};
    df.apis.db.createRecords({"table_name":"todo", "body":item},
    function (response) {
        $('#itemname').val('');
        getRecords();
    }, crudError
    );
}
You can click an item in the list to toggle its completion status. Each time you do this the function updateRecord() will be called. It calls the SDK method mergeRecords() with body containing an array of records to be updated. The database id must be included for each record being updated.
function updateRecord(id, complete) {

    var item = {"record":[
        {"id":id, "complete":complete}
    ]};
    df.apis.db.mergeRecords({"table_name":"todo", "body":item},
    function (response) {
        getRecords();
    }, crudError
    );
}
You can click the minus button next to an item to delete it from the list. Each time you do this the function deleteRecord() will be called. It calls the SDK method deleteRecords() with the ids field containing a comma-delimited list of ids to delete.
function deleteRecord(id) {

    df.apis.db.deleteRecords({"table_name":"todo", "ids":id},
    function (response) {
        getRecords();
    }, crudError
    );
}
Hopefully this gives you a basic understanding of how to use the SDK for CRUD operations. You can experiment directly with the API using the Live API Docs tab in the admin console of your DSP. You can use your developer tools to examine each API request in detail. Please post any questions or comments and we’ll get back to you.

Mobile App Development with CORs and Pub/Sub

 

JasonSykesThe ability to build apps from your desktop or local server is a huge plus for developers using their own debug tools, test suites, etc. With our latest Version 1.0.4 release, we now support these use cases plus the ability to run apps from outside of your DSP and still enjoy the flexibility of our REST API.

Simply log in to your DSP and click on the System Config menu item on the left hand side. On the System Config screen you’ll see the CORS Access options. Setting * will allow you to work from your desktop or anywhere. You may also use your localhost server, or another DSP or remote server you wish to have access to your data or service. Having the allowed verbs checkboxes enforces another layer of access control by locking down the entire DSP to use only the verbs checked for CORS. A future release will have CORS Access specifiable per application.

To use CORS without logging in, you’ll also have to allow for Guest Access, and create a Role for guest user access. This way you can limit any guest access to only database tables or services you see fit.

system config

If you’d like to pass authentication instead of having a guest role, consult our Swagger API docs here for the latest API call for logging in a user (see the /user/session POST request). That POST will return a session_id that you can pass as a new header called X-DreamFactory-Session-Token with all your future API requests. You will also need to include the API name you created with your application as X-DreamFactory-Application-Name if you haven’t already.

We’ve also added live data through a new publication / subscription framework.

We’re using Faye running on Node. Check here for the Faye documentation as well some good advice on security.

Need to see it working? Here’s a Plunker showing our Angular Todo Application using CORS and PubSub to talk to our demo server.

To see pub/sub working on it’s own, try this real time chat app on Plunker

Enjoy! And as always, if you have any questions, please contact us!

Connecting your DSP to External File Storage

Sometimes your application needs to access to external file repositories.  We’ve added a super simple way to communicate with Amazon S3 and Windows Azure Blob Storage by setting up a Remote File Service.

In this short tutorial, we’ll show you how to quickly set up a Remote File Service.

First log into your DreamFactory Services Platform (or “DSP” for short). If you don’t have a DreamFactory account, you can sign up for one here.

Set Up a Remote File Service in 60 Seconds

Once you’re in the DSP Admin Console, click on the Services menu. Then click on the “Create New Service”

DSP services menu

Then enter information about the service.

  1. In the Type dropdown select “Remote File Storage.”
  2. Enter a Service Name. The Service Name will be displayed in the Admin Console. You can edit the service later after you’ve created it.
  3. Enter an API Name. The API Name will be used when making REST calls within the DSP.
  4. Enter a Description of the service (optional).
  5. In the Storage Name field enter the folder name where your content is located on either Amazon S3 or Windows Azure.
  6. In the Storage Type dropdown, select either Amazon S3 or Windows Azure. See the directions below for adding your S3 or Azure key credentials.
  7. Check the Active box to enable the service.

Amazon S3 Users: Add Your Amazon S3 Keys

For Amazon S3 you’ll need to provide your Access Key and Secret Key.

Amazon S3

You can find these values by logging into your AWS account and selecting Security Credentials from the My Account / Console dropdown menu.

AWS security credentials

Windows Azure Users: Add Your Windows Azure Access Keys

Windows Azure Storage requires an Account Name and Account Key.

Windows Azure

You can find these values in the Windows Azure Portal.  Once you’ve logged into your portal, select the Storage menu item, then select Manage Access Keys on the bottom of the screen.  Then copy the Storage Account Name and Primary Access Key into the Account Name and Account Key fields in your DSP.

windows azure portal

DSP access keys

Now Test Your Connection!

Now it’s time to test your connection to remote file storage. Select the Manage Files menu item in the Admin Console.  You will now see a new entry for the DSP service name you just created.

DSP admin console

Double click on the service name to view your remote files.

DSP remote files

And you’re done! Now your application can access all the remote files located in the specified Amazon S3 or Windows Azure folder.

Learn More

For more information on DreamFactory Services Platform or to set up your own Free DSP visit http://www.dreamfactory.com.

 

Why We Use Swagger For API Documentation

Nothing makes a REST API easier to use than good documentation. Well, nothing except maybe a live test environment right there in the API documentation. And help with generating client side code to use your API would be awesome too. Come to think of it, having the documentation and test environment dynamic enough to update as you on-board more services would be a major plus. Well, you get all that and more with the latest DreamFactory DSP REST API with Swagger.

What is Swagger

Because REST APIs are not auto-discoverable, it was historically difficult if not impossible for fellow developers to understand API behavior (endpoints, input parameters, output values, etc) barring the availability of custom documentation or access to the API source code. This all changed with the advent of Swagger. Provided with a JSON/YAML specification defining your API structure, Swagger can create interactive API documentation, generate client libraries in many different programming languages, and more.

Why We Use Swagger

We wanted our REST API, and that of any added web services, to be easy to understand, quick to test, and simple to use right out of the box. The Swagger framework solves our server, client, documentation and testing sandbox needs, all in a language-agnostic specification, with plenty of open-sourced server and client side resources already available to help with generation. It also comes with an open-sourced front end, the Swagger UI framework, which quickly allows developers to work with the API, giving them a clear picture of how the API responds to requests with various parameters and options. It helps us to provide what a developer needs to get an app up and running with powerful web services with little time and effort.

The interactive documentation is incredibly useful, as developers are provided with a web interface which allows for easy experimentation with the underlying API. The following example depicts DreamFactory’s Swagger UI integration for a MySQL API. The user can set one or more of the many API parameters available for this particular endpoint (retrieve records from a table), identify the table name, and when the Execute button is pressed, the response will be output in JSON format.

How DreamFactory Generates API Documentation with Swagger

Documenting static services of our API is as easy as annotating server-side code, just as you would use javadoc or phpdoc. You define the API services, including path variables, query parameters, and even members of the posted body data, along with models of any data types passed between client and server. Documentation is then easily generated in various formats (json, xml, etc) and cached for speedy access from clients.

We then generate client-side code from the resulting documentation for our API. We use this generated code in our apps, as well as, presenting them as SDK jump-starters for app developers using our platform.

At DreamFactory, we added a little twist by allowing web services to be added or removed dynamically. For any non-native service that is added to your platform’s API, a json or xml Swagger document meeting the specification can be added and/or modified to define that service, immediately making it testable from the included Swagger UI framework and usable by client-side generated code.

Using Swagger Documentation with DreamFactory

The easiest way to learn more about how DreamFactory’s Swagger integration works is by spinning up a free (no credit card required) 14 day trial at https://genie.dreamfactory.com/register. Alternatively, you can download a free on-premise version of the platform via the DreamFactory website. If you’d like to trial a commercial version of the platform, or just have questions, schedule a quick call with our team!

Shedding Development Pounds

 
BenBusseMuch has been written about how cloud computing, open source software, and programming frameworks have reduced software development costs exponentially in the last decade. As more than a few pundits have said, “anyone with a good idea and an Amazon Web Services account can now create the next Facebook!” Sure, the hyperbole is for dramatic effect, but the underlying point is true: anyone with programming skills can now create a web or mobile application faster and more cheaply than ever before. The benefits of these advances are clear in the world of startup companies and developers of consumer applications. But how do enterprises, with a far more complex set of requirements and stakeholders, benefit? How do enterprises leverage “agile” and “lean” software development practices, tools, and infrastructure to be more competitive and meet the increasingly demanding expectations of customers, business partners, and their own employees? And how does the technology shift to cloud infrastructure relate to the explosion of “bring your own device” mobile usage at work? There’s no single silver bullet answer to these questions, but several important trends are relevant to how developers of business apps should be thinking about the intersection of cloud technologies and mobile to make their projects successful.
  • HTML5 is ready for prime time as the best front-end standard for companies that need to build and deploy applications that work on desktop computers, tablets, and phones. HTML5 enables companies to write one application, instead of having to write entirely separate applications for the desktop, iOS, Android, Windows, and other mobile OS platforms.
  • Front-end mobile development toolkits like PhoneGap and Sencha and frameworks like jQuery Mobile and Angular.js will gain market influence since they complement platform- independent development across mobile and web. These toolkits enable developers to package a single application for installation on all mobile operating systems. Users can easily download and install these packaged apps onto their mobile devices just as they would a normal “native” mobile application from an app store.
  • Likewise, the mobile back-end stack for mobile applications will largely be open source,  standards-based, pre-configured, highly secure, and architected specifically for mobile usage. And the mobile back-end will not be subject to someone else’s control. You can install it on your own cloud infrastructure or data center. This is the area that we at DreamFactory are tackling. We’ve written more about this in a blog post about the evolution and design goals of the Dreamfactory Services Platform.
  • Developers will benefit enormously from both front-end and back-end simplification and standardization. Now an individual or small team of application developers can quickly create a prototype app, iterate on the design, create the application, and deliver it to end users in a production environment. Standardization also enables reusable design patterns (and reusable source code in many cases). We’ve written more about this in a blog post about the rise of the front-end developer.
  • The promise of these game changing trends is exciting but enterprises are by necessity risk averse and therefore reluctant to introduce new technologies that haven’t been battle tested in production environments. Enterprises need the flexibility and control to use the core back-end infrastructure that suits their application requirements, whether public cloud, hybrid public-private cloud, or on premise. Back-end service platforms like DreamFactory must be easily installed on the desired server infrastructure and must also integrate seamlessly with existing IT development, deployment, scaling, and security tools and practices. We’ll be writing a lot more about this topic in future posts. Stay tuned!

DreamFactory & Modus Create Case Study

 
We’re excited to introduce Mike Schwartz, software architect at Modus Create and creator of SilkJS. In this guest blog post, Mike provides a great overview of how his team built a mobile address book application powered by DreamFactory on the back-end and Sencha Touch on the client. Modus Create In this blog post, I provide a quick overview of the application we built with DreamFactory and Sencha touch. If you’re a developer, the full case study is an excellent way to learn how the DreamFactory Services Platform works in practice with a real-world example. To dig deeper, you can fork the source code to the Address Book from GitHub here. Background DreamFactory partnered with Modus Create to port and enhance an existing address book application created by Modus Create to the new DreamFactory Services Platform (DSP). The address book application is for both mobile devices and has a desktop version for administration. The existing enterprise address book application we had built called RoloDeux provided a simple, yet powerful facility to access and manage employee contact information from desktop computers or mobile devices. The Ext JS-powered desktop web application was the management console, while the mobile version, employing Sencha Touch 2 framework, was meant to be an access point only. RoloDeux enjoyed a number of custom-built widgets on top of the base libraries, including a custom ExtJS 4 component called Ext.ux.SchemaGrid. This component handles the dynamic generation of ExtJS data store, grid column definitions, form for creation and editing of records, and URLs for performing API requests to do CRUD operations. The back end sported a speedy and feature-rich SilkJS server connected to a MySQL database. The Object-Relational Mapping (ORM) implementation allows database tables to be defined as JavaScript objects. Extraneous JavaScript members in the field definitions for a Schema (table) are ignored by the ORM. This allows the programmer to associate more “interesting” information about the field that SQL syntax does not allow. When you edit the Schema definitions, the ORM automatically manages the queries to the database to alter the tables there accordingly. Project Goals A key goal of this project was to demonstrate the use of the DreamFactory API, exercise that API as it was being developed, and provide a working example application for developers to learn the DreamFactory platform. The application had to be sufficiently sophisticated to demonstrate that front end developers can develop complex client-server applications using the platform. And the entire application needed to be served on the back end by the DreamFactory Services Platform. The scope of the application was extended quite a bit to include the creation of contact groups and the organization of contacts into one or more of these groups. The focus of the project would be on the mobile implementations, including a new tablet implementation. The features of the desktop administrative application were enhanced minimally to facilitate the mobile implementations. The mobile implementations were enhanced to become fully functioning contact list management applications with the ability to create, edit, delete groups, contacts, and contact information. Conclusion The project was a resounding success! The significant update to RoloDeux using DreamFactory yielded minimum refactoring and maximum benefit on both back and front end to create a new and exciting product. The full case study describes the process of porting the application, the DSP platform, and the resulting application. You can fork the source code to the Address Book application from GitHub here. Developers may peruse the repository, check out the code, and modify it. It is licensed under the MIT License. Enjoy!

How We Got Here

 
BillAppletonBack before the turn of the century people started experimenting with new ways to make function calls across the firewall. Around this time in 1998 Dave Winer invented XML-RPC, which stands for Extensible Markup Language Remote Procedure Call. His specification introduced the concept of exchanging XML documents across a URL endpoint. The request and response would constitute a function call. These XML documents were normally exchanged with an HTTP POST transaction, and since this looked like regular web traffic the function calls sliced right through most firewalls. Tony Hong stated the XMethods website about this time, and soon there were many example services that could be called. Continue reading “How We Got Here”

Build an app using DreamFactory Services Platform

 
BenBusseThe best way to start learning about how to use DreamFactory is to jump right in and build an application! Before you get going, it’s really helpful to have an example app as a concrete starting point. Todd Appleton, senior engineer in our Atlanta office, just put together a great To Do List screencast. Todd introduces a bunch of important concepts in a simple To Do list app built with HTML, Javascript (AngularJS), and a few simple lines of CSS. There’s a lot of information packed into this short screencast. You’ll learn how to:
  • Import an application from GitHub
  • Browse application source code with the DSP file manager
  • Create database schema for your app
  • Perform basic CRUD operations with REST and JSON
  • Package your application for use in any DSP
describe the image After this primer, you’ll have enough knowledge to start building your own application. Also be sure to check out more screencasts, explore the REST API, and get familiar with our documentation. We love feedback! Let us know what you think in the comments.