Monitoring APIs with Grafana

DreamFactory, Prometheus, Docker, and Grafana
DreamFactory, Prometheus, Docker, and Grafana
Now that we have explored monitoring apis with Prometheus, lets take a look at monitoring our APIs with Grafana. You may have noticed from the previous blog that Prometheus is awesome, but takes some time to fully flesh out and their dashboards aren’t the best. Good thing Grafana specializes in data visualization. We will not even have to configure all our Prometheus queries, so let’s get started.

Monitoring APIs with Grafana

We first need to confirm everything is running, if so your output should look similar to the above image. If you are not familiar with getting everything up and running please reference our previous blog. By navigating to 127.0.0.1:3000 you should see the Grafana home page. If you are not familiar with Grafana you may be wondering where the dashboards are. They are all located in the top left dropdown where we will see 4 pre-configured dashboards. Let’s review each of them to understand what each of them does.

Docker Host Dashboard

The Docker Host Dashboard shows metrics for monitoring the usage of the server. Thanks to dockerprom(link github repo) for pre-configuring all this we can now see:
  • Server uptime, CPU idle percent, number of CPU cores, available memory, swap and storage.
  • System load average graph, running and blocked by IO processes graph, interrupts graph.
  • CPU usage graph by mode (guest, idle, iowait, irq, nice, softirq, steal, system, user).
  • Memory usage graph by distribution (used, free, buffers, cached).
  • IO usage graph (read Bps, read Bps and IO time).
  • Network usage graph by device (inbound Bps, Outbound Bps).
  • Swap usage and activity graphs.
Monitoring APIs with Grafana

Docker Containers Dashboard

The Docker Containers Dashboard shows key metrics for monitoring running containers:
  • Total containers CPU load, memory and storage usage.
  • Running containers graph, system load graph, IO usage graph.
  • Container CPU usage graph.
  • Memory usage graph.
  • Cached memory usage graph.
  • Network inbound usage graph.
  • Container network outbound usage graph.
Monitoring APIs with Grafana

Monitor Services Dashboard

The Monitor Services Dashboard shows key metrics for monitoring the containers that make up the monitoring stack:
  • Prometheus container uptime, monitoring stack total memory usage, Prometheus local storage memory chunks and series.
  • Container CPU usage graph.
  • Memory usage graph.
  • Prometheus chunks to persist and persistence urgency graphs.
  • Chunks ops and checkpoint duration graphs.
  • Samples ingested rate, target scrapes and scrape duration graphs.
  • Alerts graph.
Monitoring APIs with Grafana

Conclusion

We now have a fully functioning monitoring tool with no code. Isn’t that awesome? A no code API tool and a no code data visualization tool, it can’t get much better than this. This post has only scratched the surface with regards to what you can accomplish with these tools. If you are interested in this for a Ruby application, check out our friends over at Scout who wrote a blog about doing exactly that. If you like what you see, sign up for a free trial today or contact us for more information!

Creating a Microsoft SQL Server API in Less Than 5 minutes with DreamFactory

DreamFactory and Microsoft SQL Server
DreamFactory and Microsoft SQL Server
Do you have a ton of data sources and do not know how to expose them? Do you know you need a SQL Server API but don’t know where to begin to build it? Look no further, DreamFactory can take any database and generate a fully documented and secure REST API faster than making a sandwich. All you need is your database credentials and DreamFactory will handle the rest, instantly generating Swagger documentation and securing your API by way of API keys. Follow along with the blog or our video below!

Installing DreamFactory

In order to get started you must install DreamFactory, which is Open Source and gives you the ability to try out a numerous amount of popular databases. To spin up your own hosted environment for free click here.

Generating the SQL Server API

Now the fun part. Once you have DreamFactory up and running you will be on the Admin panel. From there you will navigate to the Services tab to connect your database for your API. For example I have selected a Microsoft SQL Server database but it is nearly identical for any database you will be connecting to. Connecting your database typically only requires filling out these 5 fields as shown below.

SQL Server API

Upon saving there will be a success window pop-up saying “Service saved successfully”. What it doesn’t tell you is all the magic it just did behind the scenes. In just that short amount of time, it generated your REST API. So now if you navigate to the API Docs tab you can see your new documentation for the API and actually interact with it via the “Try it out” button.

SQL Server API

Securing and Interacting with the API

I can end it here now that you have generated your API, but where is the fun in that? Now let’s actually see the API in action! First things first, let’s generate an API key to be paired with the API for security purposes. DreamFactory does not allow access to the API without being authenticated. Let’s navigate to the Roles tab and create a Role for our API. This Role with correspond with the API key so different users can have different privileges based off different keys.

SQL Server API

For my Role I have pointed it to the SQL Server Service we just created and told it to only allow GET calls on the endpoints. This will ensure anyone using this API key will not be able to, for example delete data from the database. We must now link this Role to an API key. If we now go to the Apps tab we can create a new API key with the corresponding Role.

SQL Server API

Once we hit save we are able to see the API key generated for our use. Just to show how it works I will be using Insomnia, a popular HTTP service, to call our DreamFactory API. I will call the customers table, passing the API key in the headers for authentication.

SQL Server API

Conclusion

As you can see I have access to our new API and how much time did that take? Way less time than building this API yourself! If you would like to find out exactly how much time and money DreamFactory can save you, check out our API calculator. Otherwise what are you waiting for? Go build your next application using DreamFactory already! If you have any questions about the platform, or just APIs in general, we’d love to hear from you! Contact us.

Creating an Almost No Code Database-backed Web Interface with DreamFactory and Tabulator

No matter the size or industry, companies everywhere grapple with solving a deceptively simple problem: displaying database data on the web in a table. This problem seems to be universal in the sense that all teams have experienced the inconvenience of passing around Excel spreadsheets, and want to instead move this data to the web. Yet these same teams often lack the programming experience required to securely retrieve the data and present it within a paginated, filterable and often searchable web interface. Fortunately, using DreamFactory and off-the-shelf software such as Tabulator or DataTables, it’s possible to create a powerful database-backed web interface such as the one presented in the following screenshot:

Example dashboard interface

If you’d like to interact with a live example head over to http://tabulator.demo.dreamfactory.com/ and get your hands dirty! In this blog post I’ll show you how easy it is to create your own database-backed web interface using DreamFactory and Tabulator with almost no code required.

Creating the Database API

DreamFactory is an API generation and management platform used by thousands of organizations around the globe. You can download our open source version from DreamFactory.com or contact us to start an on premise or hosted trial of our commercial version. The open source edition includes support for several popular database, including MySQL and PostgreSQL, whereas the commercial edition additionally supports Microsoft SQL Server and Oracle. Once installed, you can generate a database-backed API in literally minutes, secured by (at minimum) an API and role-based access controls. If you’re not familiar with this process head over to DreamFactory Academy and watch the following video, “Creating Your First Database API with DreamFactory”:
You’ll also find dozens of other videos on our Youtube channel. Once the API created, you can query the database using a standardized API URL structure such as:
https://example.com/api/v2/corporate/_table/employees
Of course, for security reasons you’ll need to additionally pass along an API key. This process is described in the aforementioned video.

Integrating the API with Tabulator

Tabulator is an open source JavaScript library used for creating interactive HTML tables. It supports data loading via a JavaScript array, JSON formatted data, or an AJAX data source. We’ll use the latter approach to load data from the DreamFactory-powered database API. To begin though, you’ll first need to install the library. If you’re looking for the easiest solution, we recommend cloning our dreamfactory-tabulator repository, located on GitHub. Alternatively a number of other installation solutions are supported (Bower, NPM, Yarn, etc); check out the Tabulator documentation for more information. Once installed, you only need to add a few lines of boilerplate code to load the database data into the Tabulator table. The following example contains just 35 lines of JavaScript code and will recreate a basic version of the interface presented in the earlier screenshot:
<!DOCTYPE html>
<html>
    <head>
        <title>Tabulator example</title>
    </head>
    <body>
        <div id="example-table"></div>

        <script>

            var table = new Tabulator("#example-table", {
                    layout:"fitData",
                    height:"100%",
                    layout:"fitColumns",
                    pagination:"local",
                    paginationSize:20,
                    paginationSizeSelector:[20, 40, 60, 80],
                    placeholder:"No Data Set",
                    columns:[
                            {title:"Employee Number", field:"emp_no"},
                            {title:"First Name", field:"first_name"},
                            {title:"Last Name", field:"last_name"},
                            {title:"Birth Date", field:"birth_date"},
                            {title:"Hire Date", field:"hire_date"},
                    ],
            });

            $(document).ready(function(){
                    var ajaxConfig = {
                            dataType: 'json',
                            headers: {
                                    "X-DreamFactory-Api-Key": '123456qwerttasdf' 
                            },
                    };

                    table.setData(
                        'https://example.com/api/v2/corporate/_table/employees', 
                        {}, 
                        ajaxConfig
                    );
            });

        </script>
</body>
</html>
Let’s breakdown some of the key syntax found in this example:
  • The div identified by the example-table id will serve as the location where Tabulator will inject the table. When the Tabulator object is created, you’ll see this div ID (#example-table) is passed in as the first argument so Tabulator knows what div to use.
  • The columns defined in the columns array should match the fields returned within the DreamFactory JSON response. In this example, each record includes five fields: emp_no, first_name, last_name, birth_date, and hire_date.
  • The ajaxConfig object defines the X-DreamFactory-Api-Key header. You’ll assign the API key generated within DreamFactory to this header.
  • The setData method identifies the URL that will be contacted to retrieve the JSON data.
Check out a more involved example in our GitHub repository to learn how to add other features!

In Summary

If you’d like to learn more about DreamFactory, and how our platform can be used to quickly generate powerful web interfaces with almost no code required, contact us at DreamFactory.com!

GET this! Seven Simple REST Client Examples for Retrieving API Data

You’ve installed DreamFactory and connected to your database, and executed a few test API calls in the API Docs interface. So what’s next? Fortunately DreamFactory has already taken care of autogenerating your API endpoints so you can skip right past that and proceed to creating a client which will talk to the DreamFactory-managed API.

In this article we will demonstrate seven simple REST client examples involving sending a GET request to an HTTP-based API using an API key for authentication. We will present examples demonstrating NodeJS, Python, Ruby, PHP, and Perl. Even if you’re not a DreamFactory user (you should check it out, our OSS version is available for free download) there’s still plenty to learn from this post! Continue reading “GET this! Seven Simple REST Client Examples for Retrieving API Data”

Creating a Geofence API Using the Haversine Formula, PHP, and DreamFactory’s Scripted API Services

As a followup to the recently published post, “Creating a Geocoder Service Using Dreamfactory and the Google Maps Geocoding API”, I thought it would be fun to continue expanding upon this theme and demonstrate how to create a geofence API using the Haversine equation and a DreamFactory PHP Scripted Service API. A geofence is a virtual perimeter surrounding a location defined by a specific pair of latitudinal and longitudinal coordinates. For example, the map overlay found in the following screenshot simulates what a 500′ radius geofence placed around the iconic Empire State Building would look like:

Displaying a geofence

These days, geofences are commonly used for a wide variety of applications, including providing customers with localized coupons once they enter a store, wildlife management, and security. In this article you’ll learn how to insert the Haversine Equation into a scripted DreamFactory service. Once done you’ll be able to issue a simple HTTP call to your DreamFactory instance, passing along two sets of coordinates in order to determine how far away a subject is from the target destination. Before doing so though, it’s worth talking about the mathematical formula that makes this all possible, known as the Haversine formula.

Introducing the Haversine Formula

Wikipedia defines defines the Haversine formula as being used to “determine the great-circle distance between two points on a sphere given their longitudes and latitudes”1. The “great-circle distance” is the shortest distance between two points as measured on the surface of a sphere, which makes it a relatively accurate solution for measuring distance between two points on Earth. This formula involves a fairly heavy dose of geometry, and so I’ll leave it to the aforementioned Wikipedia page to walk you through the mathematical details. We’ll instead focus on converting this formula into its’ PHP equivalent, and making this functionality available via a secure DreamFactory service.

Creating the Geofence API

To create the Geofence API in DreamFactory, we’ll begin by creating a new PHP Scripted Service. This is accomplished by logging into your DreamFactory instance, clicking on Services, clicking Create, and selecting Script and finally PHP (see below screenshot).

Creating a PHP Scripted Service

Assign a name, label, and description to the new service. I selected geofence, Geofence API, and Geofence API, respectively. Next, click on the Config tab, and in the text area add the following PHP code:
if ($event['request']['method'] == "GET") {

    $lat1 = (float) $event['request']['parameters']['lat1'];
    $lng1 = (float) $event['request']['parameters']['lng1'];

    $lat2 = (float) $event['request']['parameters']['lat2'];
    $lng2 = (float) $event['request']['parameters']['lng2'];

    $radius = 3959;

    $dLat = deg2rad($lat2 - $lat1);
    $dLon = deg2rad($lng2 - $lng1);

    $a = sin($dLat/2) * sin($dLat/2) + cos(deg2rad($lat1)) 
             * cos(deg2rad($lat2)) * sin($dLon/2) * sin($dLon/2);
    $c = 2 * asin(sqrt($a));
    $d = $radius * $c;

    return $d;

}
This is the Haversine formula implemented in PHP and wrapped around a GET HTTP call listener. If you want to return the distance between points in kilometers instead of miles, replace 3959 with 6371. Once saved, and after configuring an associated DreamFactory role and API key, you’ll be able to call your API like so:
/api/v2/geofence?lat1=40.748667&lng1=-73.985667&lat2=40.753706&lng2=-73.982341
The 40.748667,-73.985667 coordinates identify the location of the Empire State Building, and the -73.985667,40.753706 coordinates identify the location of the New York Public Library (5th avenue branch). The service returns a distance of just 0.3892 miles; a short walk between these two famous landmarks!

Where to From Here?

Managing these sorts of API-based services inside DreamFactory is great for many reasons. Firstly, DreamFactory-managed services are reusable, meaning you won’t have to bother with repeatedly coding the Haversine formula every time the need arises to insert this sort of functionality into an application. You can instead just quickly connect to the existing service! Second, you can optionally incorporate various advanced DreamFactory features to suit application-specific needs:
  • Use DreamFactory’s limiting feature to restrict the number of hourly/daily/monthly calls a particular user can make, opening up the opportunity to monetize your data.
  • Integrate the geofence API into another API’s workflow, allowing you to return the distance between two points alongside other response data.

Conclusion

Would you like to see this example demonstrated in person? Contact our team to arrange a demo. Or download the OSS version of DreamFactory and experiment with this feature for yourself!

  1. https://en.wikipedia.org/wiki/Haversine_formula 

API Calculator: Understanding the costs behind building an API-based application

Do you want to save money, deploy projects faster, and spend more time on developing application experiences that enthrall users? Then it’s practically a certainty APIs will play a critical role in your software development process. Accordingly, you’ll want to obtain a fairly accurate understanding of the cost and time required to building an API. But it’s important to understand that there’s much more to an API than just coding an interface to some data source such as a database. Whether you are planning on hiring a contractor or assign a new project to your team, an API calculator can help you understand the time and cost required to develop a mission-critical part of your next project. Continue reading “API Calculator: Understanding the costs behind building an API-based application”

MS SQL Server vs MySQL – Which Reigns Supreme?

RDBMS databases form the backbone of most business software solutions. When people discuss SQL (Structured Query Language), it’s in reference to an RDBMS system. Applications store all their important data there. The databases (usually) power all the searches. A good database can bring a system to a higher level. A bad database can bring a business to its knees. For any developer or enterprise embarking on a new software venture, one big question is "which database vendor should I use?". In the early days of computing, database vendors such as IBM and Oracle reigned supreme. That has changed in recent years. MySQL (open source solution recently purchased by Oracle) and Microsoft’s SQL Server have gained market share. According to a 2018 StackOverflow survey, they hold the top two rankings in SQL database usage. But which one is best for YOUR business? MySQL vs SQL Server presents a tough and complicated decision! Continue reading “MS SQL Server vs MySQL – Which Reigns Supreme?”

Creating a Geocoder Service Using DreamFactory and the Google Maps Geocoding API

There is no question DreamFactory’s native connectors have saved IT teams countless hours of development time. Yet these are almost incidental when one takes into consideration the platform’s ability to integrate with thousands of third-party REST and SOAP services, not to mention create entirely new APIs through the scripted service interface. Further, thanks to DreamFactory’s ability to leverage third-party libraries, new APIs can often be created in just a few dozen lines of code. In this tutorial you’ll learn how to create a DreamFactory-managed geocoding service using Spatie’s popular open source geocoder package. Using this service, you’ll be able to easily convert addresses into latitudinal and longitudinal coordinates, as well as perform reverse geocoding (convert coordinates into an address). Once complete, you’ll be able to use a JavaScript library such as Axios or a PHP library like Guzzle to add geocoding to your application with no additional coding required! Continue reading “Creating a Geocoder Service Using DreamFactory and the Google Maps Geocoding API”

DreamFactory and GitHub – Event Scripting Bliss

DreamFactory’s Event and Platform Scripting

The DreamFactory scripting objects enable you to create a custom delivery of data through a unified URL structure. Ok, this is a pretty well-known fact if you have spent some time digging around DreamFactory. DreamFactory allows you to link your event scripts and custom scripts to a file that is managed in your GitHub account. This eliminates the need to manually update your DreamFactory scripts when you update those scripts in your source control repo.

Continue reading “DreamFactory and GitHub – Event Scripting Bliss”

Extend Your Twitter Integration With Third-Party Libraries

DreamFactory and Twitter
DreamFactory and Twitter

Integrating Third-Party Libraries With DreamFactory Scripting

Being able to script business logic into your endpoints to further extend and customize the data response(s) and request(s) is one of the best features inside of DreamFactory. As a form of ETL data integration, DreamFactory can help you create a data lake or data warehouse, especially when combined with a service such as Xplenty. Sometimes cases arise where you need to take advantage of the robust third-party library community to help extend your API. In the example below, we are going to use a third-party library to help us connect our DreamFactory instance to a Twitter Account to post updates. We also bring in the conecpt of connecting the Twitter service to our Rapsberry Pi IoT device to post temperature information to show you how can connect multiple services through DreamFactory to manage your data needs in a simple, robust method that makes API mangement a snap.

Continue reading “Extend Your Twitter Integration With Third-Party Libraries”