DreamFactory CTO Jason Gilmore recently sat down with NoSQL Database Podcast host Matt Groves to talk about DreamFactory, NoSQL database solutions, and advanced API devops topics such as performance, profiling, and logging. Here’s a chronological breakdown of the conversation:
No matter the size or industry, companies everywhere grapple with solving a deceptively simple problem: displaying database data on the web in a table. This problem seems to be universal in the sense that all teams have experienced the inconvenience of passing around Excel spreadsheets, and want to instead move this data to the web. Yet these same teams often lack the programming experience required to securely retrieve the data and present it within a paginated, filterable and often searchable web interface.
Fortunately, using DreamFactory and off-the-shelf software such as Tabulator or DataTables, it’s possible to create a powerful database-backed web interface such as the one presented in the following screenshot:
If you’d like to interact with a live example head over to http://tabulator.demo.dreamfactory.com/ and get your hands dirty!
In this blog post I’ll show you how easy it is to create your own database-backed web interface using DreamFactory and Tabulator with almost no code required.
Creating the Database API
DreamFactory is an API generation and management platform used by thousands of organizations around the globe. You can download our open source version from DreamFactory.com or contact us to start an on premise or hosted trial of our commercial version. The open source edition includes support for several popular database, including MySQL and PostgreSQL, whereas the commercial edition additionally supports Microsoft SQL Server and Oracle.
Once installed, you can generate a database-backed API in literally minutes, secured by (at minimum) an API and role-based access controls. If you’re not familiar with this process head over to DreamFactory Academy and watch the following video, “Creating Your First Database API with DreamFactory”:
You’ll also find dozens of other videos on our Youtube channel.
Once the API created, you can query the database using a standardized API URL structure such as:
Of course, for security reasons you’ll need to additionally pass along an API key. This process is described in the aforementioned video.
Integrating the API with Tabulator
Let’s breakdown some of the key syntax found in this example:
The div identified by the example-table id will serve as the location where Tabulator will inject the table. When the Tabulator object is created, you’ll see this div ID (#example-table) is passed in as the first argument so Tabulator knows what div to use.
The columns defined in the columns array should match the fields returned within the DreamFactory JSON response. In this example, each record includes five fields: emp_no, first_name, last_name, birth_date, and hire_date.
The ajaxConfig object defines the X-DreamFactory-Api-Key header. You’ll assign the API key generated within DreamFactory to this header.
The setData method identifies the URL that will be contacted to retrieve the JSON data.
We are excited to announce availability of DreamFactory 3.0 early access release. This release is the culmination of several months of hard work, and is undoubtedly the most significant platform release in more than 2 years. This release is ready to be cloned via GitHub by way of the 3.0-beta branch. Docker users can spin up the 3.0 release made available via our Docker repository.
So what’s new in this release? Let’s review the key features.
New User Interface
The most notable enhancement is a brand new interface. While the 2.X interface works just fine, it is shall we say rather utilitarian in nature. But there’s nothing wrong with adding a little aesthetic appeal, right? So while we’ve kept the user interface layout intact (for now), you’ll find that everything looks much shinier:
By the way, the new UI release coincided with the launch of our completely revamped website (https://www.dreamfactory.com/). Be sure to check it out if you haven’t already!
Automated Linux “Genie” Installers
DreamFactory talks to a lot of data sources and third-party services, and for this reason developers have done so many amazing things with the platform. But installing and configuring all of the necessary drivers can be a pretty intimidating process. Our Docker environment goes a long way towards alleviating these issues however understandably not everybody is using Docker so we wanted to ease the installation process for other users.
The result are two (soon to be 4) new installation scripts found in the installers directory. These installer scripts are used in conjunction with Debian and Ubuntu, and install all of the dependencies you need to run DreamFactory! If you’re running one of our commercial editions, the installers will additionally assist in the installation of the Microsoft SQL Server and Oracle drivers!
Python 3 Support
The DreamFactory Platform has long supported four scripting engines, including Node.js, PHP, Python, and V8JS. Scripting support is undoubtedly one of DreamFactory’s most popular features, not only because it’s used to add business logic to API endpoints, but also because you can create entirely new APIs using the Scripted Services connector. To put the popularity of this feature into perspective, we are aware of at least two customers who are running more than 400 scripted APIs within their DreamFactory environments!
One of these customers uses Python exclusively, and requested Python 3 support not only because they wanted to take advantage of various libraries that have been ported to Python 3. Always happy to prioritize customer requests, we added Python 3 support to help them out and also because official Python 2.7 updates are slated to cease on January 1, 2020.
For users running Python 2 scripts, have no fear! We added Python 3 support, and did not remove Python 2 support. When creating a new scripted service or adding logic to and endpoint, you’ll have the option of selecting between the two versions, as depicted in the following screenshot:
Updates to the GitHub, GitLab, and Bitbucket Connectors
There’s nothing particularly exciting to talk about here, other than we’ve additionally done quite a bit of maintenance work on the source control connectors, refactoring code and upgrading the Bitbucket integration to support the latest API release.
So What’s Next?
As we move towards a production release, we’ll be fixing bugs and adding a few more features to the platform:
Restricted Administration and API Auditing capabilities. We’re certainly burying the lede by casually mentioning this feature so late in the post. This is going to be a huge new addition to the platform, and will undoubtedly warrant a blog post unto itself.
Add CentOS and Fedora genie installers. These installers have been extensively tested by customers, and so in the coming weeks we’ll be adding it alongside the Debian and Ubuntu installers.
Upgrade installation scripts’ Oracle drivers to 18.5
We’d Love Your Feedback!
We’re very excited to share this release with the DreamFactory community, and would greatly appreciate your feedback! Please download, install, and try to break the software, sending comments, complaints, and suggestions to [email protected]!
The name ‘Gavin Belson’ has earned many accolades over the years: serving as Chief Innovation Officer at Hooli, founding and co-leading Nucleus compression technology, and generally making the world a better place.
Now, DreamFactory is proud to announce that Gavin Belson has joined its advisory board, bringing with him a plethora of experience to help guide and mentor the upcoming Middle-Out Middleware launch.
Arriving just in time for DreamFactory’s new product, Gavin has been central to understanding the problems facing today’s smart developers and why they need Middle-Out Middleware. “Middleware is a method of reducing the noise and software services between the end user device and the data sources that power applications. DreamFactory has already delivered a single layer of automated middleware for developers; but we are now excited to annouce a new product that changes all of that. Unlike regulare middleware, Middle-Out Middleware structures itself in such a way that devices and data sources are aligned for maximum ‘data stroke’ efficiency” Gavin stated at his latest press announcement, appearing via hologram at Hooli. He continued, “imagine a sheath cupping and pulling data from each data source, before moving to deliver the data to each device before again pulling the device data needs and moving back to the data sources. There are many efficiency losses. This is why Middle-Out Middleware changes the game. By using the same dedicated sheath to milk the data sources and device in a single smooth motion, amazing latency synergies are gained. Quite frankly, it makes everyone happy, faster.”
Gavin’s advocacy has seen a number of Fortune 500 companies queue up to prototype just what sort of synergies this could translate into. One company’s CTO, remaining under NDA, told us “[t]his is the big release we have all been waiting for“.
Interested in trialling Middle-Out Middleware yourself? Launch is officially on 1 April 2019, and for the low low price of $3m USD p.a., your company could be charging forward with a platform unleashing the benefits that come with a Weissman score of 5.2.
What does this really mean? Contact DreamFactory to find out more!
Cleared for release through Hooli Public Relations via Gavin Belson
You’ve installed DreamFactory and connected to your database, and executed a few test API calls in the API Docs interface. So what’s next? Fortunately DreamFactory has already taken care of autogenerating your API endpoints so you can skip right past that and proceed to creating a client which will talk to the DreamFactory-managed API.
As a followup to the recently published post, “Creating a Geocoder Service Using Dreamfactory and the Google Maps Geocoding API”, I thought it would be fun to continue expanding upon this theme and demonstrate how to create a geofence API using the Haversine equation and a DreamFactory PHP Scripted Service API.
A geofence is a virtual perimeter surrounding a location defined by a specific pair of latitudinal and longitudinal coordinates. For example, the map overlay found in the following screenshot simulates what a 500′ radius geofence placed around the iconic Empire State Building would look like:
These days, geofences are commonly used for a wide variety of applications, including providing customers with localized coupons once they enter a store, wildlife management, and security.
In this article you’ll learn how to insert the Haversine Equation into a scripted DreamFactory service. Once done you’ll be able to issue a simple HTTP call to your DreamFactory instance, passing along two sets of coordinates in order to determine how far away a subject is from the target destination. Before doing so though, it’s worth talking about the mathematical formula that makes this all possible, known as the Haversine formula.
Introducing the Haversine Formula
Wikipedia defines defines the Haversine formula as being used to “determine the great-circle distance between two points on a sphere given their longitudes and latitudes”1. The “great-circle distance” is the shortest distance between two points as measured on the surface of a sphere, which makes it a relatively accurate solution for measuring distance between two points on Earth.
This formula involves a fairly heavy dose of geometry, and so I’ll leave it to the aforementioned Wikipedia page to walk you through the mathematical details. We’ll instead focus on converting this formula into its’ PHP equivalent, and making this functionality available via a secure DreamFactory service.
Creating the Geofence API
To create the Geofence API in DreamFactory, we’ll begin by creating a new PHP Scripted Service. This is accomplished by logging into your DreamFactory instance, clicking on Services, clicking Create, and selecting Script and finally PHP (see below screenshot).
Assign a name, label, and description to the new service. I selected geofence, Geofence API, and Geofence API, respectively. Next, click on the Config tab, and in the text area add the following PHP code:
This is the Haversine formula implemented in PHP and wrapped around a GET HTTP call listener. If you want to return the distance between points in kilometers instead of miles, replace 3959 with 6371. Once saved, and after configuring an associated DreamFactory role and API key, you’ll be able to call your API like so:
The 40.748667,-73.985667 coordinates identify the location of the Empire State Building, and the -73.985667,40.753706 coordinates identify the location of the New York Public Library (5th avenue branch). The service returns a distance of just 0.3892 miles; a short walk between these two famous landmarks!
Where to From Here?
Managing these sorts of API-based services inside DreamFactory is great for many reasons. Firstly, DreamFactory-managed services are reusable, meaning you won’t have to bother with repeatedly coding the Haversine formula every time the need arises to insert this sort of functionality into an application. You can instead just quickly connect to the existing service!
Second, you can optionally incorporate various advanced DreamFactory features to suit application-specific needs:
Use DreamFactory’s limiting feature to restrict the number of hourly/daily/monthly calls a particular user can make, opening up the opportunity to monetize your data.
Integrate the geofence API into another API’s workflow, allowing you to return the distance between two points alongside other response data.
Do you want to save money, deploy projects faster, and spend more time on developing application experiences that enthrall users? Then it’s practically a certainty APIs will play a critical role in your software development process.
Accordingly, you’ll want to obtain a fairly accurate understanding of the cost and time required to building an API. But it’s important to understand that there’s much more to an API than just coding an interface to some data source such as a database. Whether you are planning on hiring a contractor or assign a new project to your team, an API calculator can help you understand the time and cost required to develop a mission-critical part of your next project.
Continue reading “API Calculator: Understanding the costs behind building an API-based application”
We’ve recently launched a new site for beginner-level videos about the DreamFactory platform, and yesterday published our second video in the series. It’s titled “Merging Multiple Database API Calls with DreamFactory Data Mesh”. In this video you’ll learn how to use data mesh to merge data residing in IBM DB2 and MySQL databases and present the data within a single API response.
Head on over to DreamFactory Academy to watch the video now!
Today’s IT teams are struggling to make sense of organizational data that has been compiled piecemeal and often stored within disparate storage solutions. Often this information needs to be aggregated and presented in a unified format, yet pulling data from multiple data sources and displaying it in a coherent way can be onerous and error-prone. The challenge is compounded when the data resides in different databases, and possibly within different clouds.
To remedy this, companies often embark upon costly and time consuming data lake, data mart, and data warehouse projects. In many cases though, the IT team is simply looking for an effective solution to combine data within a single unified interface!
In this tutorial I’ll introduce you to a powerful and very popular feature of the DreamFactory platform called Data Mesh. Using Data Mesh you can create virtual relationships between two databases much in the same way you can create foreign key relationships between two database tables. We’ll walk through an example in which a MySQL database running on Amazon RDS is meshed with an IBM DB2 database running on IBM Cloud, merging the data together so it can be retrieved via a single API endpoint.
Modern business runs on software. This involves storing business data, and moving that data from place to place. In the old days, software stored the data in a siloed fashion. This meant it was only available in one place, cutoff from the rest of the world. Companies struggled to pull data from different locations. Combining it in meaningful ways was difficult. This isolated data only provided a fraction of its potential value. In modern applications, the data wants to be free. With the onset of the web, data is available everywhere. Sharing data has had an exponential effect on the power of software. Applications share data using some form of API, or Application Programming Interface. With the explosion of available APIs, it has become more difficult to manage it all. API Integration allows you to combine, track, and add new APIs.
Continue reading “API Integration – What is it? The Definitive Guide”