Synchronizing Database Data Using an API

Synchronizing database data using an API is a topic that comes up repeatedly in conversations with DreamFactory users. The reasons for doing so are many, but perhaps the most commonplace is the need to fill in data gaps found in a new database. For instance, you might wish to periodically consult a customer database and synchronize relevant data with a marketing campaign database. So how might this be accomplished?

Continue reading “Synchronizing Database Data Using an API”

Extending Git with a Custom Command

The DreamFactory development team leans heavily upon Git and GitHub for repository management. Whether it’s DreamFactory Genie, the DreamFactory website, the Getting Started with DreamFactory guide, or the platform itself, you can guarantee the code is managed within a repository. Apparently my own Git usage is such that it dominates my terminal command history, accounting for 32.689% of all commands executed (per the command zsh_stats )! Yet no matter how deep your Git expertise may be, eventually you’ll have to combine commands with shell syntax in order to achieve more complicated tasks. One such example is deleting all locally merged branches. In this tutorial you’ll learn how to do so by extending Git with a custom command.

Continue reading “Extending Git with a Custom Command”

Integrate Salesforce Data Using DreamFactory

The DreamFactory platform supports thousands of data sources, but did you know the platform’s very first offering was to integrate Salesforce data? This is perhaps not a surprise, given the enormously important role Salesforce plays within enterprises large and small. IT teams often need to migrate Salesforce data into and out of databases such as MySQL and SQL Server, not to mention create customized Salesforce dashboards. These sorts of tasks are easy to do using DreamFactory’s native Salesforce connector.

We’ve just published a new chapter within our Getting Started with DreamFactory guide titled “Integrate Salesforce Data Using DreamFactory”. In this chapter you’ll learn how to configure the Salesforce connector, retrieve accounts, retrieve a specific account, and update an account. We’ll be dramatically expanding this chapter in the coming weeks to include additional features, but wanted to make this initial set of examples available as early as possible!

Introducing DreamFactory 3

After months of work, countless user conversations, and a great team effort we’re happy to announce the release of DreamFactory 3! This is undoubtedly the most exciting release in several years, addressing a great number of customer requests and addressing numerous important enhancements.

This release is ready to be cloned via GitHub. Docker users can spin up the 3.0 release made available via our Docker repository. You can also take the new release for a test drive via our new hosted service! Head on over to https://genie.dreamfactory.com to start your free 14-day trial.

So what can you expect to find in this release? Let’s break it down!

Continue reading “Introducing DreamFactory 3”

Creating an SFTP REST API with DreamFactory

SFTP REST API

Our team regularly holds detailed technical discussions with some of the largest companies in the world. From these many discussions it’s become quite apparent that lying somewhere beneath the fancy logo and glitzy commercials you’ll find IT departments relying on tried-and-true technologies and protocols to get the job done.

One particularly prominent workhorse is SFTP. The Secure File Transfer Protocol is a secure version of FTP, and it transfers data over SSH. Because developers often desire to build features capable of securely transferring files to some server, DreamFactory offers an SFTP connector capable of generating a full-featured REST API for transferring and retrieving files and directories.

Learn more about this feature in our newly published section of the Getting Started with DreamFactory guide.

Learn About DreamFactory and NoSQL – NoSQL Database Podcast Interview

DreamFactory CTO Jason Gilmore recently sat down with NoSQL Database Podcast host Matt Groves to talk about DreamFactory, NoSQL database solutions, and advanced API devops topics such as performance, profiling, and logging. Here’s a chronological breakdown of the conversation:

  • 00:48 – Introduction to Jason
  • 02:51 – Introduction to DreamFactory
  • 12:52 – NoSQL with DreamFactory
  • 16:10 – Use Case Discussion
  • 22:42 – Addition non-database integrations/transformations
  • 30:27 – Business logic in DreamFactory
  • 35:04 – Customer 360 Use Case
  • 43:03 – “Lightning Round” of technical topics: performance, indexing, logging/monitoring, profiling
If you’d like to learn more about the DreamFactory Platform while commuting or working out at the gym, download the 53 minute podcast now!

Creating an Almost No Code Database-backed Web Interface with DreamFactory and Tabulator

No matter the size or industry, companies everywhere grapple with solving a deceptively simple problem: displaying database data on the web in a table. This problem seems to be universal in the sense that all teams have experienced the inconvenience of passing around Excel spreadsheets, and want to instead move this data to the web. Yet these same teams often lack the programming experience required to securely retrieve the data and present it within a paginated, filterable and often searchable web interface. Fortunately, using DreamFactory and off-the-shelf software such as Tabulator or DataTables, it’s possible to create a powerful database-backed web interface such as the one presented in the following screenshot:

Example dashboard interface

If you’d like to interact with a live example head over to http://tabulator.demo.dreamfactory.com/ and get your hands dirty! In this blog post I’ll show you how easy it is to create your own database-backed web interface using DreamFactory and Tabulator with almost no code required.

Creating the Database API

DreamFactory is an API generation and management platform used by thousands of organizations around the globe. You can download our open source version from DreamFactory.com or contact us to start an on premise or hosted trial of our commercial version. The open source edition includes support for several popular database, including MySQL and PostgreSQL, whereas the commercial edition additionally supports Microsoft SQL Server and Oracle. Once installed, you can generate a database-backed API in literally minutes, secured by (at minimum) an API and role-based access controls. If you’re not familiar with this process head over to DreamFactory Academy and watch the following video, “Creating Your First Database API with DreamFactory”:
You’ll also find dozens of other videos on our Youtube channel. Once the API created, you can query the database using a standardized API URL structure such as:
https://example.com/api/v2/corporate/_table/employees
Of course, for security reasons you’ll need to additionally pass along an API key. This process is described in the aforementioned video.

Integrating the API with Tabulator

Tabulator is an open source JavaScript library used for creating interactive HTML tables. It supports data loading via a JavaScript array, JSON formatted data, or an AJAX data source. We’ll use the latter approach to load data from the DreamFactory-powered database API. To begin though, you’ll first need to install the library. If you’re looking for the easiest solution, we recommend cloning our dreamfactory-tabulator repository, located on GitHub. Alternatively a number of other installation solutions are supported (Bower, NPM, Yarn, etc); check out the Tabulator documentation for more information. Once installed, you only need to add a few lines of boilerplate code to load the database data into the Tabulator table. The following example contains just 35 lines of JavaScript code and will recreate a basic version of the interface presented in the earlier screenshot:
<!DOCTYPE html>
<html>
    <head>
        <title>Tabulator example</title>
    </head>
    <body>
        <div id="example-table"></div>

        <script>

            var table = new Tabulator("#example-table", {
                    layout:"fitData",
                    height:"100%",
                    layout:"fitColumns",
                    pagination:"local",
                    paginationSize:20,
                    paginationSizeSelector:[20, 40, 60, 80],
                    placeholder:"No Data Set",
                    columns:[
                            {title:"Employee Number", field:"emp_no"},
                            {title:"First Name", field:"first_name"},
                            {title:"Last Name", field:"last_name"},
                            {title:"Birth Date", field:"birth_date"},
                            {title:"Hire Date", field:"hire_date"},
                    ],
            });

            $(document).ready(function(){
                    var ajaxConfig = {
                            dataType: 'json',
                            headers: {
                                    "X-DreamFactory-Api-Key": '123456qwerttasdf' 
                            },
                    };

                    table.setData(
                        'https://example.com/api/v2/corporate/_table/employees', 
                        {}, 
                        ajaxConfig
                    );
            });

        </script>
</body>
</html>
Let’s breakdown some of the key syntax found in this example:
  • The div identified by the example-table id will serve as the location where Tabulator will inject the table. When the Tabulator object is created, you’ll see this div ID (#example-table) is passed in as the first argument so Tabulator knows what div to use.
  • The columns defined in the columns array should match the fields returned within the DreamFactory JSON response. In this example, each record includes five fields: emp_no, first_name, last_name, birth_date, and hire_date.
  • The ajaxConfig object defines the X-DreamFactory-Api-Key header. You’ll assign the API key generated within DreamFactory to this header.
  • The setData method identifies the URL that will be contacted to retrieve the JSON data.
Check out a more involved example in our GitHub repository to learn how to add other features!

In Summary

If you’d like to learn more about DreamFactory, and how our platform can be used to quickly generate powerful web interfaces with almost no code required, contact us at DreamFactory.com!

DreamFactory 3.0 Early Access Release Now Available

We are excited to announce availability of DreamFactory 3.0 early access release. This release is the culmination of several months of hard work, and is undoubtedly the most significant platform release in more than 2 years. This release is ready to be cloned via GitHub by way of the 3.0-beta branch. Docker users can spin up the 3.0 release made available via our Docker repository. So what’s new in this release? Let’s review the key features.

New User Interface

The most notable enhancement is a brand new interface. While the 2.X interface works just fine, it is shall we say rather utilitarian in nature. But there’s nothing wrong with adding a little aesthetic appeal, right? So while we’ve kept the user interface layout intact (for now), you’ll find that everything looks much shinier:

The new DreamFactory UI

By the way, the new UI release coincided with the launch of our completely revamped website (https://www.dreamfactory.com/). Be sure to check it out if you haven’t already!

Automated Linux “Genie” Installers

DreamFactory talks to a lot of data sources and third-party services, and for this reason developers have done so many amazing things with the platform. But installing and configuring all of the necessary drivers can be a pretty intimidating process. Our Docker environment goes a long way towards alleviating these issues however understandably not everybody is using Docker so we wanted to ease the installation process for other users. The result are two (soon to be 4) new installation scripts found in the installers directory. These installer scripts are used in conjunction with Debian and Ubuntu, and install all of the dependencies you need to run DreamFactory! If you’re running one of our commercial editions, the installers will additionally assist in the installation of the Microsoft SQL Server and Oracle drivers!

Python 3 Support

The DreamFactory Platform has long supported four scripting engines, including Node.js, PHP, Python, and V8JS. Scripting support is undoubtedly one of DreamFactory’s most popular features, not only because it’s used to add business logic to API endpoints, but also because you can create entirely new APIs using the Scripted Services connector. To put the popularity of this feature into perspective, we are aware of at least two customers who are running more than 400 scripted APIs within their DreamFactory environments! One of these customers uses Python exclusively, and requested Python 3 support not only because they wanted to take advantage of various libraries that have been ported to Python 3. Always happy to prioritize customer requests, we added Python 3 support to help them out and also because official Python 2.7 updates are slated to cease on January 1, 2020. For users running Python 2 scripts, have no fear! We added Python 3 support, and did not remove Python 2 support. When creating a new scripted service or adding logic to and endpoint, you’ll have the option of selecting between the two versions, as depicted in the following screenshot:

Python 3 Support

Updates to the GitHub, GitLab, and Bitbucket Connectors

There’s nothing particularly exciting to talk about here, other than we’ve additionally done quite a bit of maintenance work on the source control connectors, refactoring code and upgrading the Bitbucket integration to support the latest API release.

So What’s Next?

As we move towards a production release, we’ll be fixing bugs and adding a few more features to the platform:
  • Restricted Administration and API Auditing capabilities. We’re certainly burying the lede by casually mentioning this feature so late in the post. This is going to be a huge new addition to the platform, and will undoubtedly warrant a blog post unto itself.
  • Add CentOS and Fedora genie installers. These installers have been extensively tested by customers, and so in the coming weeks we’ll be adding it alongside the Debian and Ubuntu installers.
  • Upgrade installation scripts’ Oracle drivers to 18.5

We’d Love Your Feedback!

We’re very excited to share this release with the DreamFactory community, and would greatly appreciate your feedback! Please download, install, and try to break the software, sending comments, complaints, and suggestions to [email protected]!

Creating a Geofence API Using the Haversine Formula, PHP, and DreamFactory’s Scripted API Services

As a followup to the recently published post, “Creating a Geocoder Service Using Dreamfactory and the Google Maps Geocoding API”, I thought it would be fun to continue expanding upon this theme and demonstrate how to create a geofence API using the Haversine equation and a DreamFactory PHP Scripted Service API. A geofence is a virtual perimeter surrounding a location defined by a specific pair of latitudinal and longitudinal coordinates. For example, the map overlay found in the following screenshot simulates what a 500′ radius geofence placed around the iconic Empire State Building would look like:

Displaying a geofence

These days, geofences are commonly used for a wide variety of applications, including providing customers with localized coupons once they enter a store, wildlife management, and security. In this article you’ll learn how to insert the Haversine Equation into a scripted DreamFactory service. Once done you’ll be able to issue a simple HTTP call to your DreamFactory instance, passing along two sets of coordinates in order to determine how far away a subject is from the target destination. Before doing so though, it’s worth talking about the mathematical formula that makes this all possible, known as the Haversine formula.

Introducing the Haversine Formula

Wikipedia defines defines the Haversine formula as being used to “determine the great-circle distance between two points on a sphere given their longitudes and latitudes”1. The “great-circle distance” is the shortest distance between two points as measured on the surface of a sphere, which makes it a relatively accurate solution for measuring distance between two points on Earth. This formula involves a fairly heavy dose of geometry, and so I’ll leave it to the aforementioned Wikipedia page to walk you through the mathematical details. We’ll instead focus on converting this formula into its’ PHP equivalent, and making this functionality available via a secure DreamFactory service.

Creating the Geofence API

To create the Geofence API in DreamFactory, we’ll begin by creating a new PHP Scripted Service. This is accomplished by logging into your DreamFactory instance, clicking on Services, clicking Create, and selecting Script and finally PHP (see below screenshot).

Creating a PHP Scripted Service

Assign a name, label, and description to the new service. I selected geofence, Geofence API, and Geofence API, respectively. Next, click on the Config tab, and in the text area add the following PHP code:
if ($event['request']['method'] == "GET") {

    $lat1 = (float) $event['request']['parameters']['lat1'];
    $lng1 = (float) $event['request']['parameters']['lng1'];

    $lat2 = (float) $event['request']['parameters']['lat2'];
    $lng2 = (float) $event['request']['parameters']['lng2'];

    $radius = 3959;

    $dLat = deg2rad($lat2 - $lat1);
    $dLon = deg2rad($lng2 - $lng1);

    $a = sin($dLat/2) * sin($dLat/2) + cos(deg2rad($lat1)) 
             * cos(deg2rad($lat2)) * sin($dLon/2) * sin($dLon/2);
    $c = 2 * asin(sqrt($a));
    $d = $radius * $c;

    return $d;

}
This is the Haversine formula implemented in PHP and wrapped around a GET HTTP call listener. If you want to return the distance between points in kilometers instead of miles, replace 3959 with 6371. Once saved, and after configuring an associated DreamFactory role and API key, you’ll be able to call your API like so:
/api/v2/geofence?lat1=40.748667&lng1=-73.985667&lat2=40.753706&lng2=-73.982341
The 40.748667,-73.985667 coordinates identify the location of the Empire State Building, and the -73.985667,40.753706 coordinates identify the location of the New York Public Library (5th avenue branch). The service returns a distance of just 0.3892 miles; a short walk between these two famous landmarks!

Where to From Here?

Managing these sorts of API-based services inside DreamFactory is great for many reasons. Firstly, DreamFactory-managed services are reusable, meaning you won’t have to bother with repeatedly coding the Haversine formula every time the need arises to insert this sort of functionality into an application. You can instead just quickly connect to the existing service! Second, you can optionally incorporate various advanced DreamFactory features to suit application-specific needs:
  • Use DreamFactory’s limiting feature to restrict the number of hourly/daily/monthly calls a particular user can make, opening up the opportunity to monetize your data.
  • Integrate the geofence API into another API’s workflow, allowing you to return the distance between two points alongside other response data.

Conclusion

Would you like to see this example demonstrated in person? Contact our team to arrange a demo. Or download the OSS version of DreamFactory and experiment with this feature for yourself!

  1. https://en.wikipedia.org/wiki/Haversine_formula 

Merging Multiple Database API Calls with DreamFactory Data Mesh

We’ve recently launched a new site for beginner-level videos about the DreamFactory platform, and yesterday published our second video in the series. It’s titled “Merging Multiple Database API Calls with DreamFactory Data Mesh”. In this video you’ll learn how to use data mesh to merge data residing in IBM DB2 and MySQL databases and present the data within a single API response.

Head on over to DreamFactory Academy to watch the video now!

https://academy.dreamfactory.com/