As all developers know, your creations can become tired, bored, and sometimes angry when forced to talk to only a single server or service. Applications are, for all intents and purposes, tiny monsters: they grow hungry to communicate with others to expand their knowledge, reach, and power. We at DreamFactory know this all too well and wanted to create an easy way to allow your monsters to feed their yearning. With the next release of the DreamFactory Services Platform™ (version 1.1.3), satisfying that need will become a whole lot easier. What makes this possible you ask? A new system service called Portal does.
Fortunately, you won't need to hold hands with your team and/or conduct a séance to open a portal. However, some simple configuration steps are required. In this post, we'll be creating a portal to Facebook and accessing the Graph API through your DSP. So read on, if you dare....Muahhahaha!
All good tales have a prologue, and this one is no different. Most remote services and APIs use OAuth to authenticate and authorize users and usage of their service. We wanted to create a seamless way for your DSP to authenticate and authorize with remote OAuth services, without forcing developers to jump through a deadly row of firey rings.
If you're not familiar with the dance that is OAuth, please take a moment to read up on it. You don't need to fully understand the workings, but the concepts are important. Here are few good sites to check:
OAuth can be used to authenticate a user and authorize various operations. The authentication part is straightforward. Your application requests access to data on a user's behalf. The user will either allow or deny this access. Easy peasy. Authorization, on the other hand, can be more complex as each service defines their own set of authorization rules, or scopes. We'll touch on that a bit later.
The important take-away here is the basic concept of a client ID and client secret. It is with these keys that the power of the portal can be unlocked.
Please note: OAuth comes in two distinct flavors: version 1 and version 2. The portal service only supports version 2 at this time. Version 1 has been deprecated for over a year. However, some services, like Twitter, still use version 1.
The new DSP portal service allows you to authenticate and communicate with these remote APIs with the same REST calls you've been making to use other DSP services. Before this is possible, however, we need to prepare your configuration with the necessary tools to open the portal properly.
There are three required items for opening up a portal on your DSP. These are:
In this tutorial, we will be opening a portal to Facebook on your DSP.
Of Note...
Before you can register an application with Facebook and get a set of keys, you must be registered as a Facebook developer. Go to the Facebook Developers web site and register as a developer if you have not already. To become a Facebook developer, you must already have a regular Facebook account. In addition, and this kinda sucks, your developer account (if different) must use your real name. You will have to have a regular old Facebook account to do this.
Getting a set of keys from Facebook is a simple process: create an app. For each application you create on the Facebook Developer console you will receive a set of keys. You could, should you choose to do so, create multiple applications on Facebook and link them to different portals on your DSP. But that is beyond the scope of this article.
Click on the "Create New App" button and you'll be presented with the new application screen as shown in figure 2.
Figure 2: New App Screen
Here you'll enter the name of your application and the category. Utility or Other are good categories for portal applications. Once you've entered the required information, press the "Continue" button. The next screen, as shown in figure 3, will reveal your application keys.
Figure 3: Your keys revealed
In order to complete the setup of your application, you'll need to enter some more information. Three fields on this screen (figure 3) are required for our purposes. These are the "Contact Email", "App Domains", and your "Site URL".
In figure 3 you'll see a completed form with some example data. This configuration is for a Free Edition of the DreamFactory Services Platform™. If you're running your DSP locally or on a remote virtual provider, you'll need to change the "App Domains" and "Site URL" to point to your DSP's domain and full URL respectively.
Once you've completed filling out this form, press the "Save Changes" button at the bottom. Once that's completed, copy the App ID and App Secret from the application page. These are your client ID and secret respectively. You'll need these to create the portal provider in the next section.
With the advent of version 1.1.3, the DreamFactory Services Platform™ has two new system tables/resources. These are called Provider and ProviderUser. The provider resource securely stores application key sets for your OAuth providers. One row per key set. The provider user resource stores your DSP's users' tokens once they've authenticated. This warrants a bit of explanation.
When you create your application on Facebook, you're merely creating a pathway for your DSP to access Facebook's APIs. However, that is only the first step towards OAuth integration. You, or one of your users, must actually authenticate using the keys obtained in the previous section. Once the authentication completes, Facebook (or any OAuth provider you choose) will return an access token. This token is then used to make API calls on behalf of the owning user, NOT the Facebook application owner. The portal will coordinate and handle this token passing for you, so not to worry.
When one of your DSP users authenticates against a portal provider and is issued an access token , this token and expiration information is stored securely in your DSP's MySQL database (in the provider user table). This frees you, the developer, from having to manage and maintain the tokens for each of your users. It also allows you some flexibility in your security and permissions configuration.
Because of the sensitive nature of client IDs, secrets, and access tokens (OH MY!); there is no "admin" screen for providers and provider users. You must use the DSP REST API to manipulate this data. This can be done with any REST utility (Postman in Chrome, or REST Client in FireFox) or from the command line with cURL. This API call is like any other REST API call you make to your DSP. For example, to get the list of providers configured in your DSP, issue a GET with no ID to https://dsp-name.cloud.dreamfactory.com/rest/system/provider as shown in figure 4.
Figure 4: Getting a list of your providers with Postman
You've undoubtedly noticed that nothing was returned in that example. That's because we haven't created any providers. So let's do that now. Issue a POST request to the same URL, but this time we need to provide a payload which includes your keys.
Figure 5: The POST request
The config_text field in the new Provider resource is designed to contain an object of data. It accepts a JSON document and you're free to store any additional settings or configuration information for the provider you're creating. It is then securely encrypted and stored as a blob in the Provider table. Just include the additional properties and values in your POST/PUT payloads. Of note, however, is that each type of provider (i.e. OAuth1, OAuth2, OpenId, etc.) have a default set of required configuration settings. These are defined in the authentication library we developed for this purpose. It's called Oasys and available on GitHub. For that reason, we strongly suggest that you namespace your settings within the config_text object. Your domain name works well if nothing else:
{
"id": 1,
"config_text": {
"client_id": "abc",
"client_secret": "xyz",
"redirect_uri": "https://somewhere.com/",
"mydomain": {
"settings": {
"a": 1,
"b": 2,
"c": 3
}
}
}
}
Listing 1: Example payload with extra settings for a provider
To retrieve these values from Javascript, they can be dereferenced like any other object property: data.config_text.mydomain.settings.a would contain the value of 1. All config_text data is returned in an object when pulled from the API as shown in listing 2.
After you POST the text in figure 5 (with your keys obviously) you'll see the result as shown in figure 6. Don't use the keys shown in this article as they will not be available once this article is published.
Figure 6: The POST response. Your new provider is now ready
What we did was create a new "provider" resource with an API name of fbportal . This is the name you will use to access your portal. But before you, or your users, can use the portal, you must authenticate. To do this, simply fire up your DSP, log in, and hit the portal URL http[s]://[dsp-name]/rest/portal/<portal-api-name>/me?app_name=blog. Replacing dsp-name.cloud.dreamfactory.com with the URL to your DSP.
Here's an example using cURL from the command line:
$ curl -X GET https://dsp-name.cloud.dreamfactory.com/rest/portal?app_name=admin
{"resource":[{"id":1,"api_name":"fbportal","provider_name":"Facebook","config_text":
{"client_id":"1411182705780165","client_secret":"053556cdf3960c9b521e35c1d7e8fe40"}}]}
If you have Python installed and the json.tool , you can have the output optionally prettified as follows:
$ curl -s GET https://dsp-name.cloud.dreamfactory.com/rest/portal?app_name=admin | python -m json.tool
{
"resource": [
{
"api_name": "fbportal",
"config_text": {
"client_id": "1411182705780165",
"client_secret": "053556cdf3960c9b521e35c1d7e8fe40"
},
"id": 1,
"provider_name": "Facebook"
}
]
}
There's nothing spooky about the portal service. It merely relays calls from your DSP out to the remote provider's API. The magical part is that the access token (or possibly other keys issued to the actual user performing the authentication) is then retrieved by your DSP and secure stored into the database in the df_sys_provider_user table. It never sees the light of the internet's fiber foundation. This is a really cool feature because it allows you, the developer, to create a single application that will essentially create/maintain users local to your DSP for you automatically. In addition, it will allow you (in the case of this blog post) to access any of their Facebook information via the Graph API.
The flow works as follows (assuming the provider has been created):
https://localhost:8888/rest/portal/fbportal/me?app_name=admin for instance)data.success === false) and contains an additional field called detail.location which is the URL to which the user should be redirected to authenticate.To facilitate the portal service and the storage of provider and provider-user level keys, two new tables were added to the base DSP installation. The details of these files, or any other system table, can be found in the /config/schema/system_schema.json file off of your project root directory.
Figure 8 depicts an ERD of the new tables and how they relate to the existing df_sys_user table, which is, your DSP's user/login table.
The two new tables are df_sys_provider and df_sys_provider_user. The former, storing provider-level settings and credentials. The latter stores user-level credentials and settings. The database will enforce referential integrity. That is to say, if you delete a provider, all user tokens associated with that provider will be deleted permanently.
Lumbergh Says...
Yeah... so... I'm going to have to go ahead, and ask you, to make a backup -- or snapshot if you're using a hosted DSP -- of your database BEFORE goofing with sensitive information. Since this data is encrypted, any stored keys will be difficult (if not impossible) to recover.
Yes, even we at DreamFactory can't see your keys once the product is released.
This table encrypts and stores local portal and remote login providers for your DSP only. No other DSPs may access or utilize these providers.
This table encrypts and stores local portal and remote login provider credentials for each user who has authenticated against the portal. As you'll recall, once you've created your Facebook application, users still need to log into Facebook with their own Facebook accounts and allow of their personal information to your DSP. We'll get into the mechanics of that in a bit.
Figure 8: New Tables
Figure 8 is an ERD of where the Provider and Provider User tables fit and their relationships.
We also have added two new system resources: provider and provider_user. These can be manipulated like any other system resource via the /rest/system/provider and /rest/system/provider_user endpoints.
Read on to part 2 - OAuth Tutorial Part 2